AMD's 192-core Turin chip doubles Llama 3 speed, no GPU required

The era of running serious LLM inference without a GPU just got a lot more interesting. AMD’s 5th Gen EPYC “Turin” processors are delivering roughly double the throughput of the previous Genoa chips on Meta’s Llama 3.1 8B model, according to fresh benchmarks from Hugging Face. We’re talking a 192-core, 384-thread monster that makes CPU-only deployment genuinely viable for a broader set of tasks.

Hugging Face partnered with AMD to validate the entire ecosystem—Transformers, Optimum, and the new ZenDNN 5.0 plugin for PyTorch—ahead of Turin becoming widely available. They stress-tested Llama 3.1 8B across five real-world scenarios: summarization, chatbot work, translation, essay writing, and live captioning. The tests used bfloat16 precision and a multi-instance setup, running multiple model copies in parallel with 32 physical cores allocated per instance. Across batch sizes of 16 and 32, Turin consistently posted a 2X speedup in decode throughput compared to Genoa.

What’s particularly clever here is the integration of AMD’s ZenDNN plugin with PyTorch’s torch.compile. The plugin performs graph-level optimizations on the model’s intermediate representation, squeezing extra performance out of the silicon without requiring developers to change their code. Hugging Face is playing this one close to the chest—they got early access to benchmark, but the actual ZenDNN 5.0 release and a pre-configured Dockerfile for easy reproduction are both marked “coming soon.”

For teams currently paying a premium for GPU instances just to serve moderate-traffic chatbots or summarization pipelines, this could reshape the cost equation. CPU instances are cheaper and more abundant. A 2X throughput jump doesn’t erase the GPU advantage for massive batch processing or training, but it narrows the gap significantly for inference tasks where latency isn’t measured in single-digit milliseconds. The benchmark code and Docker setup will drop publicly once AMD gives the green light—and that’s when we’ll see how well these numbers hold up outside a controlled lab environment. The real test is whether community developers can replicate that 2X speedup on their own messy, real-world inference stacks.