Apple and Hugging Face just made Stable Diffusion XL dramatically more practical on Apple Silicon. The headline number: a 71% size reduction, from 4.8 GB down to 1.4 GB, using a technique they’re calling mixed-bit palettization. That puts SDXL within reach of local Mac users who don’t have CUDA hardware or don’t want to pay for cloud inference.
The new Core ML port covers the base SDXL pipeline, published as two variants on the Hugging Face Hub. One is a straight conversion with no quantization. The other swaps the UNet for a mixed-bit recipe that lands at an effective 4.5 bits per parameter. Apple’s own quality measurements, expressed as PSNR scores, show the compressed version at 75.8 versus 82.2 for the original float16 model — a real but controlled drop. “In our opinion quality is still great,” the announcement reads, which is the kind of qualified claim you want from engineers who actually looked at the outputs.
What’s genuinely novel here is the method itself. Instead of applying a uniform 6-bit quantization across every layer, the team measures per-layer sensitivity by comparing PSNR degradation against the float16 baseline. Layers that degrade badly at 2 bits get bumped to 4, then 8, and critical layers can stay at 16-bit. The result is a recipe — a layer-by-layer bit allocation — that can hit aggressive averages like 2.8 bits while keeping quality loss within a chosen budget.
There are caveats. The refiner stage hasn’t been ported yet, and the current implementation uses the original attention mechanism, which is tuned for CPU+GPU compute units rather than the ANE. Performance will vary by device, though Apple published figures for several Macs. The real test will be whether fine-tuned community models convert cleanly through the updated conversion repo. Given how many SDXL fine-tunes have flooded Civitai and the Hub since July, that’s where this tooling will prove its worth — or reveal its limits.
