Apple just made running Stable Diffusion on iPhones, iPads, and Macs significantly faster and lighter with a pair of updates to Core ML and the coremltools conversion package. The headline feature is 6-bit palettization, a quantization method that compresses model weights from 16-bit floating point down to 6 bits per parameter. The trick borrows from computer graphics: weights are replaced with indexes into a small color-style table, which slashes storage and download size. But here’s where it gets interesting. Previous Core ML versions would uncompress those weights when the model loaded from disk, meaning memory usage matched the full uncompressed size. The new approach keeps weights compressed and converts them on the fly as inference moves through each layer. Counterintuitively, that’s faster, because memory transfers sit in the critical path of execution. Moving less data beats moving uncompressed data, even with the extra decompression overhead.
Apple’s ml-stable-diffusion repo, originally released last December, has been updated with quantization support during conversion using a —quantize-nbits flag ranging from 8 bits down to 2. The company recommends 6-bit as the sweet spot, where precision loss stays minimal while speed and memory gains stay substantial. The update also includes a new attention layer optimization called SPLIT_EINSUM_V2 that splits query sequences into 512-token chunks to avoid creating large intermediate tensors. Apple claims this alone improves performance by 10% to 30% on devices with a Neural Engine.
Apple has converted all four official Stable Diffusion models to the new format and pushed them to Hugging Face Hub in both Core ML format and zip archives for native app integration. To use the 6-bit models, developers need the beta versions of iOS/iPadOS 17 or macOS 14 Sonoma, which contain the updated Core ML framework. Registered developers can grab them now, while everyone else waits a few weeks for the public beta.
This move lands at a moment when on-device ML is having a genuine moment. The explosion of interest in running Stable Diffusion and LLMs locally, driven by privacy concerns and API cost fatigue, has developers hunting for any efficiency gains. Core ML optimizations are the kind of under-the-hood work that rarely makes headlines but quietly determines whether a model feels snappy or sluggish on a phone. The 6-bit palettization approach specifically mirrors the direction the broader AI inference world has been heading, squeezing more capability into consumer hardware without requiring users to buy new devices.
