Benchmark contamination has quietly become one of the dirtiest open secrets in AI evaluation. Models train on the internet. The internet contains test questions. The result is a trust gap that makes every impressive scorecard a little suspect. Google DeepMind is now trying to close that gap with something genuinely novel: a double-blind evaluation that locks external test prompts in a cryptographic box, preventing a Gemini model from ever seeing them in advance.
The pilot, run with the Singapore AI Safety Institute and other partners, uses Google Cloud’s Confidential Space to create a setup where neither side has to blink first. Evaluators keep their prompts private; Google keeps its model weights private. A Gemini Flash Lite model gets tested against confidential benchmarks, and cryptographic verification ensures no one peeked. DeepMind’s pitch is that technical protection finally replaces the zero-logging promises and contractual handshakes that have papered over this problem for years.
The stakes are highest in sensitive domains. Government agencies and cybersecurity evaluators have historically faced a brutal tradeoff: hand over your test data, or hand over your model. The ARC-AGI evaluation of Anthropic’s Fable 5 got delayed precisely because of this standoff, with Anthropic enforcing a 30-day data retention policy on its strongest models. Google’s approach sidesteps that deadlock entirely.
Whether this becomes an industry standard depends on whether other labs adopt it voluntarily. Google is publishing the methodology and results in a technical report, which is the right move. But a cryptographic fix from one lab doesn’t solve the systemic problem. What it does do is prove that double-blind evaluation is technically feasible at the frontier-model scale. That’s a bigger deal than any single benchmark score could ever be.
