Google finally decided to play in the open-source sandbox. Gemma, a new family of open LLMs built on Gemini research, dropped today with four models total: 7B and 2B parameter versions, each with base and instruction-tuned variants. The 7B model is the headline act — it scores comparably to Mistral 7B on the Hugging Face LLM Leaderboard, which puts it among the strongest open models in that weight class. The 2B version is positioned for CPU and on-device use, though it doesn’t match the benchmark scores of similarly-sized competitors like Microsoft’s Phi-2.
Hugging Face has gone all-in on the launch. The models are live on the Hub with Transformers 4.38 integration, safetensors support, and compatibility with Flash Attention 2, bitsandbytes quantization, and PEFT for efficient fine-tuning. There’s also a torch.compile() path with CUDA graphs that promises roughly 4x faster inference. Google and Hugging Face clearly coordinated this release — the integration depth suggests weeks of joint work.
Here’s what’s conspicuously absent: the technical report covers training and evaluation for the base models, but Google skips over dataset composition details and says nothing about the fine-tuning data or hyperparameters used for the instruction-tuned versions. We know the training data included web documents, code, and mathematical texts, filtered for CSAM content and PII. That’s about it. For a company positioning these as production-ready open models, the transparency gap is notable — especially when competitors like Meta have been more forthcoming about Llama’s training pipeline.
The instruct models use a strict turn-based prompt format with <start_of_turn> and <end_of_turn> markers. Get the format wrong and performance degrades. A month after launch, Google pushed an updated instruct version with better coding, factuality, and multi-turn quality, plus a fix for the models’ tendency to start responses with “Sure,” — a small but telling sign that real-world usage is already shaping the roadmap.
