Google's C4 VMs Slash AI Costs 70% Over C3 in Intel-Hugging Face GPT Test

Google Cloud’s new C4 virtual machines, powered by Intel’s Xeon 6 ‘Granite Rapids’ processors, deliver a 1.7x improvement in total cost of ownership for running OpenAI’s open-source GPT model when compared to the previous-generation C3 instances. That’s the headline finding from a joint benchmarking exercise with Intel and Hugging Face, which focused on text generation performance for the 120-billion-parameter Mixture of Experts (MoE) model. The tests showed a 1.4x to 1.7x boost in throughput per virtual CPU per dollar, helped in part by the C4’s lower per-hour price.

The performance edge isn’t just about newer silicon. Intel and Hugging Face collaborated on a key optimization (PR #40304) in the Transformers library that eliminates redundant computation in MoE models. Before the fix, every expert in the model processed all tokens, wasting floating-point operations. Now, each expert only runs on the tokens it’s specifically routed to. This is a big deal for CPU-based inference, where efficiency directly translates to cost savings. The benchmarking pitted a 144-vCPU C4 instance against a 176-vCPU C3 instance, using static KV caches and SDPA attention for consistent, deterministic runs.

MoE architectures are attractive because they let you scale model capacity without a linear increase in compute costs. Yet the practical hurdle has always been inference efficiency. This benchmark proves that with the right software-hardware pairing—Intel’s latest Xeon 6 and an optimized Transformers library—running a massive 120B-parameter model on CPUs is not just viable, it’s economically compelling. The tests measured steady-state decoding across batch sizes from 1 to 64, generating 1024-token sequences from 1024-token prompts.

Google is clearly positioning the C4 as a workhorse for inference-heavy AI workloads. The 70% TCO improvement metric is a composite one, blending raw throughput gains with the instance’s lower list price. For engineering teams watching their cloud bills, this shifts the calculus on where to deploy large MoE models. The implication is that GPU-less inference for large language models is maturing fast, but the real question is whether these CPU gains hold up under the unpredictable latency demands of production traffic, not just clean benchmarks.