Artificial Analysis has rolled out version 4.2 of its Intelligence Index, a move that appears to address growing skepticism about how it evaluated OpenAI’s latest model, GPT-6 Astra. Previous iterations of the index placed Astra roughly on par with its predecessor, a result that clashed sharply with other major evaluations. Epoch AI, for instance, ranked Astra first among 267 models with 169 points, while ARC-AGI-3 data indicated a significant performance leap. This discrepancy created confusion in the market, prompting the benchmarking firm to intervene before its planned major update later this year.
The revised numbers tell a different story. GPT-6 Astra now registers a four-point gain over its predecessor, securing second place on the leaderboard. Anthropic’s Claude Fable 5.1 retains the top spot, followed by Astra and then Meta’s offerings. Beyond raw scores, the updated analysis highlights Astra’s efficiency, noting it uses fewer tokens per task than any other frontier model. When evaluating cost-to-performance ratios, the field tightens significantly, with Anthropic, OpenAI, Meta, and Zhipu AI all sharing the lead. This suggests that while raw intelligence varies, economic viability is becoming a crowded race among top-tier providers.
To prevent future gaming and improve real-world relevance, the index introduces two new benchmarks: AA-Briefcase for knowledge work and GDP.pdf for document analysis. Conversely, GPQA-Diamond has been dropped because models have effectively solved it. Private test data now accounts for 40 percent of the weighting, a strategic shift designed to shield evaluations from manipulation. Artificial Analysis also corrected scoring errors across several existing benchmarks to stabilize results. The firm admitted it initially delayed updates to maintain stability during major launches but found the leaderboard moving too quickly to wait.
Version 5 of the index has been in development for eight months and will arrive in stages. This interim patch serves as a necessary correction in a landscape where static benchmarks rapidly lose validity. By increasing the weight of private data and retiring solved tests, Artificial Analysis is attempting to stay ahead of model capabilities rather than chasing them. Whether these changes will hold up against the next wave of releases remains to be seen, but the urgency of this update underscores how volatile current AI performance metrics have become.
