OpenAI has finally published hard numbers for Jalapeño, its first custom inference chip, and the results are substantial enough to make Nvidia’s data center team pay attention. Across three public models — GPT-OSS 120B, DeepSeek R1, and Kimi K2.5 1T — the chip delivered 1.5 to 1.9 times more AI work per watt at peak throughput compared to leading commercial systems. Latency improvements were even more dramatic: 1.7 to 3.6 times lower end-to-end, jumping to 2.1 to 4.1 times higher performance for highly interactive workloads.
What’s notable here is that OpenAI benchmarked against external models, not just its own. DeepSeek R1 and Kimi K2.5 are competitors’ architectures, and Jalapeño handled them just as efficiently. The company used InferenceX, a public benchmark from SemiAnalysis, rather than cooking up proprietary tests — a move that lends credibility to the claims. The chip is rated at 700 watts but stayed at or below 550 watts sustained during testing, which suggests real-world efficiency is better than the spec sheet implies.
The architecture itself is built around a simple observation: language model inference has distinct bottlenecks at different phases. Prefill is compute-bound, decode is memory-bandwidth-bound, and communication between chips can leave processing units idle. OpenAI’s answer was to minimize data movement by keeping model state — including the KV cache — local and activating the right mix of compute, memory, and networking for each phase. The network is integral, with a large enough domain to keep entire workloads within one connected system.
OpenAI says the gains widen further on its own frontier models, though it didn’t publish those specific numbers. That’s the part worth watching. If Jalapeño’s advantage scales with model size, the chip could give OpenAI a structural cost advantage as models grow — and make the company less dependent on Nvidia’s roadmap. The company frames this as the first generation of a multigenerational platform, with production ramping in the months ahead. The inference market just got more interesting.
