The instinct when building a coding agent is to obsess over which model to use. LangChain’s Terminal-Bench experiment suggests that’s the wrong question entirely. Keeping the model identical and changing only the harness—the scaffolding that runs the agent loop, manages tools, and handles context—moved a coding agent from roughly 30th place into the top five. That’s a staggering jump for something most teams treat as a deployment detail.
Paul Iusztin’s open-source course, Building a Coding Agent From Scratch, builds a Python agent called Decode that makes this concrete. The core is a headless harness with no interface of its own—just the loop every agent shares: the LLM picks an action, a tool executes, the observation feeds back. In Decode, that agent definition is only about 20 lines of Pydantic AI code. Everything else—memory, skills, sandboxing, permissions, compaction—is harness. The same architecture shows up in Claude Code’s leaked source, where the core loop runs roughly 150 lines.
Decode then splits into three run modes, each with a different latency profile and therefore a different inference provider. Interactive mode streams tokens to a terminal UI and is latency-bound, so it belongs on a low-latency hosted API. Remote mode runs headless on a server, fanning out a backlog of tickets to parallel harnesses that each produce their own PR; throughput per dollar is what matters there. Async mode sits between the two—a live session hands work to a queue and returns immediately, billing like batch rather than chat.
The cost arithmetic is stark. Processing 1,000 documents at 30,000 input tokens each runs about $97 at frontier API rates, even with prompt caching, because every document has a different prefix. Batched on a serverless GPU at 3,000 tokens per second, the same work costs roughly $13. The reverse is just as sharp: leave an interactive agent idle overnight on a single H200, and ten idle hours add about $45 to the bill. The rule of thumb that falls out is simple—interactive work pays per token because a human is waiting, while offline and async work pays per GPU-hour because throughput is the objective and idle time is the enemy. That’s the kind of architecture decision most teams are not making explicitly, and it’s costing them real money.
