VDR-2B-Multi obliterates OCR pipelines with 500K-sample multilingual visual search

Forget OCR and chunking strategies. The new open-source vdr-2b-multi-v1 model from LlamaIndex and MrLight lets you search screenshots of documents directly, and it speaks five languages out of the box.

Trained on a massive, self-made dataset of 500,000 query-image pairs spanning English, French, German, Italian, and Spanish, this model encodes page screenshots into dense vectors. You can query a German contract using an Italian phrase without translating anything first. The training data itself is now the largest open-source synthetic dataset for visual document retrieval, dwarfing the previous gold standard—the ColPali training set—by a factor of ten.

What makes this particularly useful is the efficiency play. By using just 768 image patches, the English-only sibling model actually outperforms its base version running on 2,560 patches. That’s a 3x inference speed bump and a massive drop in VRAM consumption. Combine that with Matryoshka Representation Learning, and you can slash vector size by 3x while retaining 98% of embedding quality. For anyone running retrieval at scale, that cuts storage costs and accelerates search latency simultaneously.

The team didn’t just scrape blindly. They built the dataset by generating topic-specific queries in each language, scraping PDFs that matched, and then using a document layout analysis model to ensure a balanced mix of text-only, visual-only, and mixed pages. Queries were synthesized by Gemini 1.5 Pro and Qwen2-VL-72B, forced to distinguish between general and specific questions so only the sharpest examples made it into training. The result is a model that performs better on real-world messy documents, not just clean academic benchmarks. It’s a pragmatic step toward making dense retrieval truly plug-and-play for multilingual archives—no preprocessing required.