Nvidia published research Friday that should reframe how the industry thinks about AI agents. Using Claude Opus 5 with a custom harness — the software wrapper that handles tools, memory, and runtime rules — researchers achieved a perfect 100% score on ARC-AGI-3, a benchmark of interactive 2D games with no instructions. The same model scored just 30% without that harness, which was still the best result among all models tested.
The gap is the story. ARC-AGI-3 has been a sore spot for OpenAI, whose models scored below 10% and prompted the company to run its own research last month. OpenAI found that tweaking two harness settings tripled its scores. Yet no model came close to Nvidia’s result. The difference: Nvidia’s harness included a supervisor component — what Adel El Hallak, vice president of product in Nvidia’s AI unit, described as acting “almost like a CEO to nudge the agent when it goes off direction.” That supervisor steps in when the main agent gets stuck or starts retracing dead ends.
This matters for anyone building on these systems. Microsoft’s April research tested 19 LLMs on long-horizon document editing and found all of them, including frontier models, produced work riddled with errors. Agents have also been caught deleting user files or turning to collusion and hacking. The model alone wasn’t the problem. The harness was.
El Hallak frames this as an argument for open agent stacks. “We believe in having an open agent stack — where you have control across the harness,” he said, connecting Nvidia’s position to reports of OpenAI slowing model training after security breaches. Databricks CEO Ali Ghodsi made a similar point in July: the same model on different harnesses can double your costs. Nvidia isn’t shipping a new product here — the harness, called Agentic Variation Operators, is built from pieces under its Nemo brand, much of it open. The signal to users: you have more knobs to turn than you think.
