Salamandra 7B gains vision via SigLIP, trained on 6.1M multilingual samples

The Language Technologies Lab has given eyes to its Salamandra 7B model, unveiling Visual Salamandra — a multimodal system that actually delivers on European linguistic diversity instead of just paying it lip service. They’ve bolted Google’s SigLIP-So400m vision encoder onto the instructed LLM via a two-layer MLP projector, using a late-fusion architecture that keeps the visual and textual streams independent until they need to talk.

The training pipeline is methodical, not magic. Phase one freezes everything except that projector, forcing it to learn how to map image features into the language model’s latent space. Only then do they unfreeze the full stack for high-quality pretraining on OCR-heavy and re-captioned datasets. Instruction tuning follows, then a final multimodal phase that throws single images, multiple images, and video at the model alongside 842,000 text-only examples to prevent catastrophic forgetting. Altogether, 6.1 million instruction-tuning instances went into this.

What genuinely sets this apart from the parade of multimodal models released every week is the language coverage. Most vision-language systems are English-first with a few translations bolted on. Visual Salamandra was built from the ground up to handle European languages that typically get ignored in this space. The training corpus pulls from AI2D, Cambrian, and LLaVA Next datasets, but the team emphasizes representation for those under-resourced languages throughout the instruction-tuning process.

Applications run the standard multimodal gamut — VQA, OCR, document understanding, mathematical reasoning — but the late-fusion design and linguistic breadth hint at uses in government services, cultural heritage digitization, and cross-border European applications where language can’t be an afterthought. The model is open under Apache 2.0, though the Lab is refreshingly candid about limitations: it hallucinates on ambiguous images, chokes on dense document layouts, and shouldn’t be trusted in high-stakes settings without human oversight. That honesty matters when every other model launch reads like a press release claiming to solve everything short of world hunger.