Hugging Face's LeRobot shrinks robot datasets 86% with video codecs

Robot datasets are monstrously bloated. The standard practice of storing visual data as individual PNG frames creates files where 95% of the content is redundant — the same robot arm, same lab bench, frame after frame. Hugging Face’s LeRobot team just proposed a format that ditches images for video, and the results are exactly what you’d hope for.

Their new LeRobotDataset format leans on modern video codecs to exploit both spatial and temporal compression. Instead of saving every pixel of every frame, the encoder identifies keyframes (I-frames), stores those as spatially compressed images, and then records only the differences between successive frames (P-frames and B-frames). The result: datasets that average 14% of their original size. In the best-case scenario, they hit a staggering 0.2% of the original footprint. A 20GB dataset collapses to roughly 2.8GB on average, or a single gigabyte at a 20:1 compression ratio.

But compression is useless if it kills training speed. The team benchmarked decoding performance and found something genuinely practical. When pulling a single random frame — the nominal case for many training loops — video decoding speed is comparable to loading a compressed PNG. The real payoff comes when you need consecutive frames, which is common in imitation learning setups that rely on observation histories. There, video decoding runs at 25% to 50% of the time it would take to load the equivalent PNGs. You get smaller storage and faster sequential access.

The format ships with native integration to the Hugging Face Hub, along with visualization tools you can try in their Spaces right now. For the robotics community — which has been stuck trading data between labs in a mess of hdf5, zarr, pickle, and tar archives — a standardized, shareable format that doesn’t require a storage array just to hold a few hundred demonstration trajectories is a meaningful step forward. What’s left unsaid is how this changes dataset release norms. If a lab can publish a 1GB dataset instead of a 50GB one, the barrier to reproducing results drops considerably. That alone might matter more than the technical elegance of the codec pipeline.