OpenAI's API now auto-caches prompts, slashing costs 50%

OpenAI just made life a little cheaper for anyone building on its API. The company rolled out automatic Prompt Caching, a feature that gives developers a 50% discount on input tokens the model has recently seen. No code changes required. If you’re reusing the same context across calls — think codebase edits or sprawling chatbot conversations — the system quietly kicks in on prompts longer than 1,024 tokens.

The mechanics are straightforward. The API caches the longest prefix of a prompt that’s been previously computed, starting at that 1,024-token threshold and climbing in 128-token increments. When you reuse a prompt with a common prefix, the discount applies automatically. You’ll spot a cached_tokens value in the usage field of the API response, a small breadcrumb that signals your bill just got lighter. The feature works on the latest versions of GPT-4o, GPT-4o mini, o1-preview, o1-mini, and their fine-tuned variants.

There is a clock on those caches, though. They typically evaporate after 5 to 10 minutes of inactivity and are guaranteed to be wiped within an hour of last use. For developers hammering the API in rapid iteration cycles, that window is generous enough to matter. For sporadic queries, maybe less so. OpenAI stressed the usual enterprise privacy commitments apply here — caches aren’t shared between organizations, which should ease the minds of security-conscious teams.

This isn’t flashy. It’s plumbing. But the kind of plumbing that quietly reshapes how developers architect applications. A 50% cut on repeated context is real money at scale, and the zero-integration design removes the usual friction of cost optimization. The move also puts quiet pressure on competitors like Anthropic and Google, who have their own caching stories to tell, to match the seamlessness. For now, it’s a straightforward win for anyone building persistent, context-heavy AI features.