OpenAI speeds up agentic workflows 40% with WebSockets

OpenAI just made agent loops in the Responses API 40% faster by slashing API overhead instead of just relying on faster inference. The core problem? Even though new models like GPT-5.3-Codex-Spark can now generate nearly 1,000 tokens per second—up from 65—the old approach of making dozens of synchronous API calls for each agent step meant users were stuck waiting on CPU-side processing, not the GPUs. The solution focused on caching rendered tokens and model configs in memory, cutting out unnecessary network hops (like image processing calls), and improving the safety classifier pipeline to speed up content checks. But the biggest win was moving to persistent WebSocket connections instead of a chain of individual requests. This reduces the cumulative overhead from each back-and-forth in an agentic loop, making the experience feel snappier for tools like Codex that scan code, make edits, and run tests. Why it matters: As inference speeds skyrocket, the bottleneck in AI agents shifts from model latency to system architecture—so optimizing the communication layer is becoming a competitive advantage for real-time use cases.