OpenAI’s GPT-6 Astra is getting wildly different report cards depending on who’s grading. Epoch AI puts it in first place among 267 models with 169 points across more than 50 benchmarks. Artificial Analysis, meanwhile, scores it at 61 — dead even with its predecessor and behind Claude Fable 5.1 at 66. Same model, two labs, opposite conclusions. That kind of divergence tells you more about benchmark design than it does about the model itself.
The real story is ARC-AGI-3. Astra hit 62.7 percent on the test, which drops models into unfamiliar game worlds with no instructions and forces them to learn by trial and error. That’s up from 7.78 percent for GPT-5.6 Sol and 30.16 for Claude Opus 5. François Chollet at ARC Prize called the progress “2x faster” than expected and moved his AGI forecast forward. The 99.9 percent figure OpenAI reported came from its own harness, which keeps reasoning chains alive between requests — a setup that ran 3.66 times faster and used 49 percent fewer tokens. ARC Prize only counts the 62.7 percent from the internal harness for fair comparisons, and that distinction echoes the same vendor-harness dispute they had with Sol.
Here’s the genuinely odd part: with Astra, more reasoning actually lowers cost. On the standard ARC scaffold, running with no reasoning cost $49,791 and scored 35.2 percent. At maximum reasoning, cost dropped to $26,098 while the score climbed to 62.7 percent. The model solves games in fewer moves, which means fewer calls and fewer tokens. You also get a strange outlier — the “low” reasoning level scored 17.5 percent, worse than no reasoning at all. ARC Prize didn’t explain it, but Astra’s architecture appears to loop processing internally before generating the first token, which might make the low setting redundant or even counterproductive.
Efficiency shows up elsewhere too. Astra needs only a third of the compute steps Sol used and a fifth of what Opus 5 requires. On coding tasks, it matches Claude Fable 5’s score while costing less than half per task. But it’s not all wins. The hallucination rate on AA-Omniscience dropped from 92 to 51 percent — still, one in two answers is wrong on that test. And the model lost about 80 Elo points on GDPval-AA v2 while slipping on banking support and long-context reasoning. On FrontierMath Erdős, Astra was the only model to solve two of 68 open problems with Lean-verified proofs at $300 per attempt, though three more solutions came from non-standardized runs that burned over $220,000 and don’t count. The efficiency gains are real, but the benchmark chaos suggests we’re now measuring models with yardsticks that don’t agree on what an inch is.
