Hugging Face slashes data startup times 100x, streaming now beats local SSDs

Hugging Face just solved one of the most tedious bottlenecks in machine learning: waiting hours for terabyte-scale datasets to download before training even begins. The company overhauled the streaming backend in its datasets library, and the result is a 100x reduction in startup requests and a 2x bump in sample throughput. If you’ve ever stared at a progress bar while your GPUs sat idle, this is for you.

The fix came from real pain. While training the SmolLM3 model, engineers at Hugging Face hit a wall—a 3-hour wait to download data before every single run. They’d previously relied on streaming from Amazon S3 for projects like SmolVLM, but shifting to Hub-based streaming for their nanoVLM work immediately triggered a new problem: over 100,000 requests in under a minute, which got their IP banned. The culprit? Every DataLoader worker was independently resolving the list of data files, creating a storm of redundant API calls.

The team attacked this in two phases. For startup, they introduced a persistent data files cache shared across all workers. Now, only the first worker fetches the file list from the Hub; the rest read from a local cache, virtually eliminating the request flood. They also optimized the resolution logic to bundle API calls more efficiently. For streaming speed, they enabled prefetching for Parquet datasets, which fetches the next data chunk in the background while the model processes the current one. Advanced users can even fine-tune buffer block sizes and prefetch limits for maximum I/O control.

The performance claims are bold: the company says streaming now outruns local SSDs when training on a 64xH100 cluster with 256 download workers. A key piece of the puzzle is Xet, Hugging Face’s deduplication-based storage system that transfers repeated data only once. The changes are fully backwards-compatible—adding streaming=True to your existing load_dataset call gets you all these improvements with zero code changes.