Artificial Analysis just dropped its first Search Index, and it’s the kind of standardized testing the AI agent space has desperately needed. Seven search API providers — Parallel, Exa, Firecrawl, You.com, Tavily, Keenable, and Brave — were put through identical agent setups using GPT-5.6 Luna on Stirrup, the firm’s open-source framework. Only the search provider changed between runs, with 25 attempts per task to smooth out variance.
The methodology pulls from three benchmarks weighted equally: DeepSearchQA’s 900 multi-query research questions, a 200-question BrowseComp subset requiring multi-step browsing for obscure facts, and AA-Omniscience’s 600 questions spanning six knowledge domains. A tool-free baseline shows what the model can do without any search assistance, giving the numbers a useful floor.
The most interesting finding isn’t about raw quality scores — it’s about how search quality cascades into everything else. Parallel Search (advanced) cut token consumption by over 40 percent compared to the Basic version, simply because the model didn’t have to keep re-querying to find what it needed. Per-task search costs actually went up with the better tier, but total cost dropped from $0.11 to $0.084 per task. That’s the kind of counterintuitive result that makes benchmarks worth reading.
Speed tells a similar story. Parallel Search (turbo) posted the fastest per-query response at 0.51 seconds versus 1.03 seconds for Basic, but its lower quality score (67 vs. 73) forced the agent into more search passes. Total time per task ended up roughly identical. Artificial Analysis flagged Parallel, Firecrawl, and Parallel (turbo) as hitting the best cost-performance balance, though the field is open for other providers to join. This is the first time anyone has benchmarked agent search APIs with this level of rigor, and it arrives just as enterprises are starting to wire retrieval into production agent workflows. The fact that faster queries can produce slower tasks is exactly the kind of insight that only shows up when you test systems end-to-end rather than components in isolation.
