AI Pulse by Inblix

Topic: data-deduplication

2 articles

Explore our coverage of data-deduplication — 2 curated articles, summaries, and related resources from the Inblix archive.

Hugging Face migrates 4.5 TB to Xet storage, turning 13-minute uploads into 0.1 seconds — Inblix summary
Research

Hugging Face migrates 4.5 TB to Xet storage, turning 13-minute uploads into 0.1 seconds

Hugging Face Blog · Mar 18, 2025 · 3 min read

Hugging Face just took its biggest swing yet at the file-size problem plaguing AI development. On February 20th, the Xe...

A single row change in a 2GB Parquet file just cost 230MB—here’s the fix Hugging Face found — Inblix summary
Research

A single row change in a 2GB Parquet file just cost 230MB—here’s the fix Hugging Face found

Hugging Face Blog · Oct 5, 2024 · 2 min read

Most of us assume that tweaking a single row in a massive dataset should be a lightweight operation. Hugging Face engin...