0

Run AI workloads on any cloud, store on Hugging Face: zero-egress storage with SkyPilot

https://huggingface.co/blog/skypilot-hf-storage(huggingface.co)
SkyPilot now integrates with Hugging Face Storage, allowing AI workloads to run on any cloud while models and datasets remain on the Hub. This solution eliminates egress fees, as Hugging Face does not charge for reading data, which avoids the typical cross-cloud transfer tax when using GPUs on different providers. Users can mount Hugging Face Buckets or repositories directly into a SkyPilot job using a simple `hf://` URL and their existing token. This setup enables teams to utilize available GPUs on any cloud without being locked into the vendor where their data is stored. The underlying Xet-backed storage also provides deduplication for efficient handling of model checkpoints and variants.
0 pointsby will222 hours ago

Comments (0)

No comments yet. Be the first to comment!

Want to join the discussion?