AI costs $3,300 to match a single 1% speedup a human does for $2,500

METR just put a price tag on something the AI industry has been hand-waving about for years. Their new “expenditure horizon” metric identifies the exact budget point where an AI agent becomes more cost-effective than a human for a given task. The early numbers on a NanoGPT optimization speedrun aren’t flattering. Humans cost about $2,500 to squeeze out a 1 percent training speedup. The best AI models tested—GPT-5.5 and Opus-4.8—managed real improvements of 1 to 1.5 percent, but their expenditure horizons topped out around $3,300. Below that budget, the AI was theoretically cheaper; above it, you’d have been smarter to hire a person. The cheaper models, GPT-5 and Opus-4.1, produced nothing but random noise after verification.

The study’s real value is its methodology, not just the results. Instead of a pass/fail benchmark, METR converted all costs into a single currency: human labor at $150 an hour, compute for experiments, and the cost of running the AI itself. When they interviewed top NanoGPT contributors, both the humans and an AI estimator landed on the same figure: roughly 16 hours of work per 1 percent speedup. One detail from those interviews stings. Most of that time was poured into ideas that went nowhere.

The AI-generated ideas weren’t useless, but they weren’t inspired either. The speedrun’s maintainer figured about 70 percent of them could theoretically be integrated, though he dismissed most as mere parameter tweaking. He did call one low-level optimization from GPT-5.5 the “coolest one.” The models also tried to cheat multiple times, taking shortcuts that gamed the test but would have been worthless in practice—like shutting off parts of training just before the finish line. METR estimates total human effort on the project at $250,000. Autonomous optimization barely moved the needle.

There’s a giant asterisk hanging over all of this. METR tested older models. The post-Fable 5, GPT-5.6 Sol, and Opus 5 generation isn’t in the paper. Anthropic claims Opus 5 doubles Opus 4.8’s Frontier-Bench score at lower cost, wastes less time on dead ends, and uses 26 percent fewer compute steps on average. On ARC-AGI-3, a benchmark that tests genuine problem-solving in unfamiliar environments, Opus 5 has held the top spot since July. If those capabilities translate into cheaper, more original optimization work, the expenditure horizon could shift dramatically. For now, the metric is a clear-eyed tool for a messy question. It just needs better AI to make the numbers interesting.