How an innocent $0.05 AI task silently balloons into a $5.00 money pit

Building a single-turn LLM wrapper is a weekend project. Keeping an autonomous agent from silently bankrupting your infrastructure over a six-month deployment is a different problem entirely. The core issue driving this quiet cost explosion is a confusion between state and context. State is the minimum set of facts needed to move a task forward. Context is the full, verbose transcript of everything that’s happened so far. Most frameworks dump the latter into the former by default, and that’s when a $0.05 automation quietly metastasizes into a $5.00 infinite loop.

The first and most pervasive trap is the O(N²) context accumulation tax. By step 20 of a workflow, the model is re-reading every message from steps 1 through 19, and you’re paying for the privilege. The fix isn’t free, though. Compress the history too aggressively with a rolling summary and you get context amnesia—the agent forgets a critical parameter from step 2, hallucinates a replacement in step 8, and cascades into a chain of failed downstream tool calls. Prompt caching can help freeze the prefix state so you only pay for the delta, but it requires a disciplined architecture that knows what’s static.

Things get actively worse when failures enter the mix. Unbounded retry loops on stale state mean a standard ReAct agent that hits a 400 Bad Request will append the raw error trace to its bloated context and then re-read the entire disaster on every subsequent retry. The smart play here is a circuit breaker at the orchestrator level. Strip the failed trajectory before re-prompting, but don’t leave the model blind—inject a deterministic, minimal failure heuristic like “Tool X failed because parameter Y was missing” instead of the raw stack trace. That preserves corrective information without the token bloat.

The third trap is unfiltered tool payload bloat, and it’s shockingly common. An agent queries a REST API, gets back a 2KB JSON response full of HTTP headers, pagination metadata, and nested objects, and the entire raw payload gets shoved into the next LLM call. Worse, many frameworks duplicate the same massive system prompt on every turn, burning tokens for instructions the model has already internalized. The fix is a ruthless payload filtering middleware that extracts only the semantically relevant fields, paired with runtime prompt injection that delivers instructions once and then relies on the state object for dynamic updates. These aren’t hypothetical edge cases; they’re the architectural patterns that separate a proof-of-concept from something you can actually afford to run.