OpenAI and 29 others map 10 ways to verify AI claims

A sprawling new report co-authored by 58 researchers across 30 organizations — including OpenAI, Mila, and the Centre for the Future of Intelligence — lays out 10 concrete mechanisms designed to make AI developers actually prove what they claim about their systems. It’s a direct response to a growing credibility gap. Lots of companies publish lofty ethics principles, but outsiders have almost no way to check whether any of it is real. That ambiguity isn’t just an academic problem; the authors argue it could fuel competitive corner-cutting and increase social harms.

The mechanisms are split across institutional, software, and hardware categories. On the institutional side, the report pushes for third-party auditing, red teaming exercises, and even bug bounties specifically for bias and safety flaws. The software recommendations get more technical — audit trails, interpretability research, and privacy-preserving machine learning tools that come with standardized performance benchmarks. On hardware, the authors want to see secure chips for AI accelerators and highly detailed, public accounting of the computing power behind major projects.

What’s genuinely interesting here is the audience. The report frames these mechanisms as useful for everyone from a user translating sensitive documents to a regulator investigating an autonomous vehicle crash. One pointed question it raises: how can an academic conduct impartial risk research when they lack industry-scale computing resources? The answer, the report suggests, requires a big funding boost from governments.

The report doesn’t claim to have solved verification. The authors call it a starting point for dialogue and say OpenAI itself plans to adopt several of the mechanisms. They’re actively looking for collaborators, and the full document comes with caveats about how much each mechanism can actually deliver. It’s an unusually practical menu of options from a field that often prefers abstract principles.