Hugging Face’s datasets library has quietly evolved from a text-focused tool into something that handles images natively. The new Image feature type accepts file paths, raw bytes, NumPy arrays, or PIL objects, and it pairs with the ImageFolder loader to pull in datasets directly from a URL. That’s exactly what this experiment does with a 10,000-image sample from the British Library’s Digitised Books collection, spanning roughly 1510 to 1900.
The problem with this collection is metadata, or rather the lack of it. These images were extracted from OCR’d books, so curators know which volume each came from but not what’s actually depicted. Flickr tagging and machine learning classification projects have helped, but keyword search only gets you so far. What if you could search by visual similarity instead?
That’s where FAISS comes in. The datasets library has supported FAISS indexing for text retrieval for a while, and the same mechanism works for images once you’ve generated embeddings. The approach uses sentence_transformers to create dense vector representations of each image, then adds a FAISS index to the dataset for efficient similarity search. The rich library is used to inspect the Image feature’s internals, revealing it stores data as a struct with bytes and path fields underneath.
The demo is deliberately small-scale, but the implications are significant. Cultural heritage institutions sit on millions of poorly-described images, and semantic visual search could unlock access without requiring manual annotation. If the same pipeline scales to the full British Library collection, researchers could trace visual motifs across centuries of printed books. The bottleneck remains embedding quality and compute cost, but the tooling is now surprisingly accessible.
