OpenAI's CUA sets benchmark records but still fumbles 62% of computer tasks

OpenAI pulled back the curtain on Computer-Using Agent, the model powering its new Operator research preview. CUA ditches specialized APIs entirely. Instead, it consumes raw pixels from a screen and manipulates a virtual mouse and keyboard—the same universal interface a person uses. The company is positioning this as a fundamental shift, letting AI navigate the buttons, menus, and text fields of any graphical user interface without custom integrations for each site or operating system.

The benchmark numbers tell a story of genuine progress and very real limits. On web-based tasks, CUA hits an 87% success rate on WebVoyager, a test involving live sites like Amazon and GitHub. But those tasks skew simple. On WebArena, a harder test using self-hosted e-commerce and forum sites, that figure drops to 58.1%. The real gut check is OSWorld, a benchmark for controlling full operating systems like Ubuntu and Windows. There, CUA manages just 38.1%—against a human baseline of 72.4%. The model does show test-time scaling, improving when given more steps to work with, but the gap is still a chasm.

Under the hood, CUA runs an iterative perception-reasoning-action loop. It takes screenshots, thinks through next moves using chain-of-thought reasoning, and then clicks, scrolls, or types. That inner monologue lets it self-correct when something goes sideways. Safety handrails are built in for sensitive actions like logins or CAPTCHAs, where the model explicitly asks for user confirmation. OpenAI is leaning hard on the “research preview” label here, rolling Operator out first to Pro tier users in the U.S. to gather real-world feedback before a wider release.

What’s genuinely new isn’t just the vision-plus-reasoning combo—it’s the insistence on using the same tools humans do. No APIs, no shortcuts. If CUA can’t see it on screen, it can’t touch it. That generality is the whole pitch, and the reason the OSWorld number stings. A 38% success rate means the model fails nearly two out of three full computer-use tasks. That’s not a product ready to replace your admin assistant. It’s a research milestone with a very long tail of edge cases left to solve. The question isn’t whether this approach works—it’s how many more rounds of reinforcement learning it takes before the failure rate stops being the headline.