AI Agents Had $3,000 and Six Days to Write a Top-Tier Research Paper. They Failed.

The industry keeps telling us recursive self-improvement is right around the corner. A new study from Princeton researchers says maybe pump the brakes. Peter Kirgis and Sayash Kapoor led a multi-institution team that put Anthropic’s Claude Opus 4.8 through what they call “shadow evaluation”—handing the agent a research question from a real, unpublished NeurIPS 2026 submission and asking it to produce work worthy of acceptance. The catch: since the papers weren’t public, the model couldn’t just memorize the answer from training data.

The setup was generous. Six days, $3,000 in API credits, GPU access, virtual computers, open web access. The agents did what agents do well—they reviewed literature, ran hundreds of experiments, and compiled results. But when the original paper authors graded the output as they would any conference submission, both papers got rejected. Kapoor didn’t mince words: the agents were “unambiguously bad at carrying out the research itself.” They tested hypotheses on tiny synthetic datasets, wrote unintelligibly, and contributed nothing novel. The engineering was fine. The science wasn’t.

The failures are instructive. Agents committed to unpromising approaches too quickly and couldn’t backtrack when things went sideways. They rejected ambitious hypotheses that actually resembled what the human authors started with—based on very limited data. When subagents or external review tools offered feedback, the orchestrator didn’t revise its methodology; it just narrowed claims and added caveats. Resource management was also poor. They struggled to follow basic instructions about time allocation and paper length.

This matters because open-ended research—choosing hypotheses, knowing when to abandon an approach, exercising judgment about what evidence settles a question—is precisely what recursive self-improvement would require. The study suggests current agents can execute tasks but can’t yet decide which tasks are worth executing. That’s the gap between automation and innovation, and it’s wider than the hype cycle would have you believe. One bright spot: no reward hacking. The agents didn’t fudge data, and the orchestrator caught subagent hallucinations. So at least they fail honestly. For now.