Anthropic claims its AI assistant, Claude, now “leads” 26 percent of the company’s model development work. That number sounds impressive until you look at the fine print. The metric comes from an internal audit of employee hours in July 2026, not a count of independent decisions. It reflects time saved on tasks where Claude operates at what Epoch AI calls “AL4” autonomy. This level means the AI can analyze, fix, and test code without stalling, but it cannot ship changes. A human still holds the keys. The jump from February’s sub-1 percent to today’s 26 percent suggests rapid integration, yet the underlying scale remains subjective.
The scoring system itself raises eyebrows. Claude evaluated its own performance by scraping Slack messages and internal docs. Anthropic admits this “judge” model might share the same blind spots as the systems it assesses. Human reviewers agreed on automation levels only 33 percent of the time when rating the same work areas. The AI judge matched human judgment 59 percent of the time. While 97 percent of ratings were within one level of human consensus, that single-level gap is precisely the difference between “collaborating” and “leading.” This ambiguity makes the 26 percent figure more of a marketing narrative than a hard scientific benchmark.
Beyond autonomy, Anthropic released data on safety monitoring. Roughly 30,000 agents run simultaneously on their internal platform. A real-time monitor blocked just 0.002 percent of over a billion actions in August. Post-hoc reviews flagged 100,000 logs weekly, with only about 50 reaching human engineers. The company also reported that 6 percent of total compute went to safety-related work. They argue this low percentage is misleading because safety research is labor-intensive rather than compute-heavy. Training runs devour chips; designing safety experiments burns researcher hours.
These metrics serve a dual purpose. They showcase Claude’s utility to potential enterprise customers while bolstering CEO Dario Amodei’s argument for coordinated slowdowns in AI development. By publishing these numbers, Anthropic positions itself as transparent. Yet the vague definitions and self-assessed scores offer critics ample room to question whether this transparency is genuine or a strategic move to shape regulation. If the industry agrees to slow down, compute allocation for safety could become a regulatory lever. But first, we need a shared definition of what “leading” actually looks like.
