Perplexity's WANDR benchmark exposes a brutal truth: research agents can't yet assemble 170,000 evidence-backed records

Most AI benchmarks are multiple-choice tests. They’re tidy, they’re narrow, and they don’t resemble the sprawling information-gathering projects that make up actual knowledge work. Perplexity decided to fix that with WANDR, a new open benchmark that’s part stress test, part reality check. It asks agents to do something deceptively simple: build large, structured collections of entities and back every single claim with a verified web source. No narrative summaries. No getting by on a few impressive examples. Just raw, verifiable completeness.

The benchmark is built from 500 tasks modeled after real production usage, not synthetic prompts. Each task defines a hierarchy—like ‘find 70 companies, then a specific executive at each, then an authoritative URL for that appointment.’ The median task demands 50 members and 245 total records. Across the entire suite, an agent would need to produce 170,495 source-backed records to achieve a perfect score. That scale is the point. WANDR is the broader counterpart to Perplexity’s earlier DRACO benchmark, which focused on the quality of a single long-form report. This one measures whether an agent can even build the evidence file in the first place.

Perplexity ran six production systems against these 500 tasks, and the results are humbling. Its own Search as Code system led the pack, but no one is close to solving this. At the highest effort setting, Perplexity managed a soft F1 score of just 0.447. The cost to get there ranged from three cents to nearly $325 per task, a four-order-of-magnitude spread. Digging into the failures reveals a clear structural problem. Discovering the right entities in the first place is the initial choke point; top-level discovery completion rates ranged from 61% to 95%. But the real killer is evidence quality. For Perplexity’s own system, 41.4% of the pages it found were missing a substantive requirement, and a staggering 57.5% of cited text excerpts didn’t fully support the claim they were meant to prove. Its soft F1 score collapses from 0.531 to 0.363 when you switch from a simple retrieval check to a full evidence verification.

What’s clear is that partial progress is easy and complete coverage is brutally hard. Every system showed higher soft recall than soft precision, meaning they’re better at the initial hunt than the meticulous verification. Deeper task hierarchies compound the problem geometrically, as each branch in the qualification tree adds a new point of failure. WANDR doesn’t just hand out a score; its grader re-fetches every cited URL and independently verifies that the excerpts genuinely support every requirement. That makes it a genuine test of an agent’s ability to do the kind of scut work a junior analyst might handle today. The gap between finding a usable page and proving a claim with it remains wide, and for now, that gap is where most research agents still live.