Introducing Storage Buckets on the Hugging Face Hub

Hugging Face has introduced Storage Buckets, a new S3-like object storage solution designed for production machine learning workflows. Unlike version-controlled Git repositories, Buckets are made for mutable, intermediate artifacts like model checkpoints, processed data, and logs. Built on the Xet chunk-based storage backend, this feature efficiently handles ML artifacts by deduplicating shared content, which reduces storage costs and speeds up data transfers. Users can manage Buckets through a command-line interface or Python library, and a pre-warming feature moves data closer to compute resources for better performance.