Microsoft’s ONNX Runtime team has delivered substantial performance gains for Stable Diffusion’s fastest text-to-image models, with throughput improvements reaching as high as 229% for SDXL Turbo and 120% for SD Turbo compared to PyTorch. The benchmarks, run on an A100-SXM4-80GB and an RTX 4090, tested various batch size and step count combinations using fp16 models with the LCM Scheduler at 512x512 resolution.
The optimizations live in ONNX Runtime’s CUDA and TensorRT execution providers. The CUDA provider showed particularly strong results for dynamic shapes, while both providers outpaced PyTorch across every tested configuration. The team notes that PyTorch’s static-shape pipeline had channel-last memory format and torch.compile with reduce-overhead mode applied, making the comparison more rigorous than a stock setup.
Olive, Microsoft’s hardware-aware model optimization tool, generated the optimized ONNX files now hosted on Hugging Face. One quirk worth knowing: SDXL Turbo’s original VAE can’t run in fp16 precision, so the team used the sdxl-vae-fp16-fix variant in testing. The decoded images are close enough for most purposes, though not pixel-identical to the original VAE output.
Beyond raw performance, ONNX Runtime opens doors for developers working outside Python. OnnxStack, a .NET library, builds on Microsoft’s earlier C# tutorial to offer Stable Diffusion inference capabilities across multiple models. Oracle has also shipped a Java sample that runs Stable Diffusion on top of ONNX Runtime. The ONNX Runtime Extension for Automatic1111’s SD WebUI exists too, though it’s only fully optimized for Stable Diffusion 1.5 so far — SD Turbo and SDXL Turbo work, but their performance optimizations are still in progress.
