7 asyncio patterns agents need, and the one that still burns your tokens

Firing off a single AI agent is straightforward. Coordinating a fleet of them with Python’s asyncio without tanking your event loop or your API budget is a different skillset entirely. The primitives are there, but the failure modes are sneaky—silently swallowed exceptions, memory leaks you won’t notice until the OOM killer shows up, and the maddening realization that you’re still paying for a generation you cancelled half a minute ago.

There’s a topology for every problem. Fire-and-forget looks cheap until a background agent fails and nobody hears it scream. Strict scatter-gather with asyncio.gather() is the classic fan-out, but it forces your whole operation to wait on the slowest model—one agent generating a novel-length response bottlenecks everything else. For Python 3.11+ shops, Task Groups are the cleaner upgrade, though their eagerness to cancel sibling tasks on a single rate-limit error means you’d better wrap those coroutines in solid retry logic.

The patterns get more interesting when you move beyond simple parallelism. A producer-consumer setup with asyncio.Queue decouples your work generation from processing, letting you scale consumers independently. But skip the maxsize parameter and you’ve built a memory bomb: a fast producer and slow consumers will inflate that queue until your process runs out of RAM. Semaphores are the practical backbone for limiting concurrent access to rate-limited APIs, but they count connections, not tokens. You can cap at 10 concurrent requests and still blow through a provider’s tokens-per-minute ceiling if all ten are churning through massive context windows.

Then there’s the pattern that sounds like a clever optimization and often isn’t: speculative execution. Racing a fast, cheap model against a slower, more capable one and cancelling the loser the moment a winner emerges trades compute for speed. The ugly catch is that cancellation only drops your local connection. The provider’s servers keep generating, and those tokens keep billing. You pay for every losing horse in the race. Pipeline chaining rounds out the toolbox, linking agents sequentially where each stage consumes the output of the last—a natural fit for RAG or multi-step reasoning flows. The real lesson across all seven patterns is that Python’s async machinery gives you enough rope to hang yourself. Knowing where the gallows are is what keeps a production system alive.