Google's SigLIP 2 adds a decoder and self-distillation to build vision encoders that actually know where things are

Google just released SigLIP 2, a family of multilingual vision-language encoders that makes the original SigLIP look like a rough draft. The core idea is simple but effective: they took the original sigmoid loss training and bolted on a few extra objectives that force the model to pay attention to spatial relationships and fine-grained details.

The most straightforward upgrade is a text decoder slapped onto the standard vision-text encoder setup. This decoder isn’t just generating holistic captions; it’s tasked with predicting bounding box coordinates from region descriptions and the reverse. That cross-modal ping-pong forces the vision encoder to develop a real sense of locality—it has to know where things are, not just that they’re there. The paper shows this creates substantially stronger representations for tasks like zero-shot classification and image-text retrieval across all model scales, from Base up to Giant.

Then comes the clever bit from self-supervised learning. After 80% of training is complete, they layer on a self-distillation routine with a Global-Local loss and a Masked Prediction loss. A student model sees a partial crop or a heavily masked image and must match the feature representations of a teacher model that saw the full picture. The teacher is just an exponential moving average of the student. It’s an expensive step they delay to save compute, but it significantly sharpens the model’s local semantics without wrecking the core alignment.

Perhaps the most practical wrinkle is the NaFlex variant, which cribs ideas from FlexiViT and NaViT to handle dynamic resolutions and native aspect ratios during a single forward pass. This isn’t just an academic benchmark win; for anyone building OCR tools or document understanding pipelines where you can’t just square-crop everything to death, a single model that handles variable aspect ratios gracefully is a genuine workflow improvement. You can already find the full model suite, from fixed-resolution checkpoints to the flexible NaFlex versions, available for download.