Shopify used AI agents to refactor its codebase — and they wanted the same things humans do

Forget the sci-fi fantasies of sentient code. Shopify’s experiment with AI-driven code refactoring delivered a surprisingly grounded result: the agents performed best when the code was clean, contracts were explicit, and feedback was immediate. In other words, they thrived under the exact same conditions a senior developer demands before touching a legacy monolith.

The project involved deploying AI agents to modernize parts of Shopify’s sprawling codebase. Engineers didn’t just throw the agents at a mess and hope for the best. Instead, they discovered that clear interfaces and well-structured modules were a prerequisite for the agents to make safe, useful changes. When the code was tangled and implicit, the agents hallucinated fixes or produced brittle patches that broke under scrutiny.

What’s notable here isn’t the technology itself — plenty of shops are experimenting with coding agents. It’s the brutal, practical constraint this surfaces. If you’re banking on AI to clean up your technical debt, you might be in for a rude awakening. The tech debt itself becomes a blocker for the very tool you bought to fix it. The agents didn’t magically understand intent; they relied on the same signals — type hints, test coverage, naming conventions — that make a codebase maintainable for people.

This flips the ROI pitch on its head. The companies that will extract real value from coding agents aren’t the ones drowning in spaghetti code. They’re the ones who’ve already done the unglamorous work of writing comprehensible software. For everyone else, Shopify’s findings suggest you’ll need to invest in human-led cleanup before the bots can meaningfully pitch in. AI doesn’t let you skip the fundamentals. It just raises the cost of ignoring them.