OpenAI finally showed its hand at Hot Chips, and the numbers are genuinely disruptive. The company’s first custom silicon, a general-purpose inference accelerator called Jalapeño, reportedly delivers 1.5x to 1.9x more AI work per watt than Nvidia’s Blackwell at peak throughput. Latency tells an even starker story: end-to-end response times came in 1.7x to 3.6x lower than the best commercially available systems across three tested models, and interactive workloads hit 2.1x to 4.1x higher performance.
The benchmark results come from SemiAnalysis’s public InferenceX suite, with OpenAI supplying the numbers and SemiAnalysis verifying some runs on-site. Models tested included GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. Jalapeño pushed roughly 1,400 tokens per second per user on GPT-OSS and over 700 tokens per second on DeepSeek R1 with a single concurrent request — and did so without speculative decoding or multi-token prediction, techniques that some comparison systems leaned on. SemiAnalysis CEO Dylan Patel didn’t mince words: “Usually first generation chips aren’t competitive, but OpenAI is beating Nvidia Blackwell and even Rubin.”
There’s a fairer fight to consider. Since both Jalapeño and Nvidia’s newer Vera Rubin platform use HBM4 memory, SemiAnalysis argues that’s the real comparison — and Jalapeño still edges ahead on output tokens per megawatt, even though Rubin uses multi-token prediction. Total cost of ownership per token lands roughly even. Caveats apply: Nvidia and AMD have published results on larger models like DeepSeek V4 Pro that Jalapeño hasn’t been tested on, and while Rubin systems are already shipping, Jalapeño remains stuck at the engineering sample stage.
OpenAI built the chip with Broadcom in a compressed timeline — design kicked off in mid-2024, the final blueprint went to fabrication in November 2025, and only nine months separated first design from tape-out. The company used its own models to accelerate the work. This is where SemiAnalysis drops its boldest claim: “The CUDA moat is potentially dead given how fast OpenAI can bring up new models on their silicon.” That’s a big statement, but the broader AI industry has been quietly hedging on CUDA for years. What makes this different is speed — a first-generation chip from a company with no silicon track record landing competitive benchmarks this quickly. CFO Sarah Friar frames the chip as part of an integrated compute stack rather than a replacement for Nvidia, AMD, or AWS partnerships. That’s the diplomatic answer. The strategic reality is that every major OpenAI partner is also building competing silicon, and Jalapeño gives OpenAI leverage it never had before.
