Hugging Face adds 4 Dataset Search filters to tame its 350K+ datasets

Hugging Face just gave its Dataset Hub a serious upgrade, rolling out four new search filters that make finding the right dataset feel less like archaeology. The announcement covers modality, size, format, and library compatibility — all of which can be stacked with existing filters for language, task, and license.

Modality filtering is the most immediately useful addition. The Hub now auto-detects whether a dataset contains text, image, audio, tabular, time-series, 3D, video, or geospatial data based on file contents and extensions. Want something with both text and images? You can filter for that combination directly. That’s a real timesaver for multimodal model work, where hunting down paired data has historically meant cross-referencing paper appendices.

Size search is where things get interesting for anyone working with LLM-scale corpora. You can now set a minimum and maximum row count, and Hugging Face says the number is available for all datasets in supported formats. For the truly massive ones where metadata doesn’t include row counts, the platform estimates totals based on the first 5GB of content. That’s a pragmatic compromise — not exact, but close enough to filter for datasets exceeding 10 billion rows without downloading anything.

Format and library filters round out the release. The format filter acknowledges a truth that’s easy to forget: the same data in Parquet versus WebDataset versus JSON Lines has very different performance characteristics. Parquet gives you nested data and efficient analytics but requires decoding entire row groups to access a single row. WebDataset streams fastest but lacks per-file row counts, which complicates multi-node training distribution. The library filter builds on this by flagging compatibility — Dask can handle large JSON Lines files that would choke Pandas, which needs everything in memory.

This isn’t just UI polish. The Dataset Hub now hosts a staggering number of datasets, and discoverability has become the actual bottleneck. I’d compare this to what happened with model search: once the volume crossed a threshold, filters went from nice-to-have to essential infrastructure. Hugging Face also provides code snippets for loading any dataset in your preferred tool, which lowers the friction between finding data and actually using it. The company is inviting library maintainers to open discussions if they want their tools added to the compatibility list — a sign that this feature set is meant to expand beyond the initial four filters.