FLUX.2 demands 80GB VRAM but can now run on a 24GB GPU — here's how

Black Forest Labs just dropped FLUX.2, a new open image generation model that is decidedly not a simple upgrade to FLUX.1. It’s a ground-up rethinking of the architecture. The biggest immediate change is the text encoder — out go the dual encoders, replaced by a single Mistral Small 3.1 model. That simplifies the prompt embedding pipeline and lets the model stack outputs from intermediate layers, a trick the team says is more beneficial. The diffusion transformer backbone got a major overhaul, too. Time and guidance modulation parameters are now shared across blocks, every bias parameter was stripped out entirely, and the ‘single-stream’ transformer blocks are now fully parallel, fusing QKV and feed-forward projections. The ratio has also shifted dramatically: single-stream blocks now account for ~73% of the DiT parameters, compared to ~54% in the last generation.

That architectural ambition comes with a brutal hardware reality. Running the full model without any offloading chews through over 80GB of VRAM. For most people, that’s a non-starter. Even with CPU offloading enabled on an H100, it still needs around 62GB. That’s the cost of a larger DiT and a heavy text encoder like Mistral3 Small running in concert. It’s a genuinely demanding workload, not just marketing fluff about needing the latest hardware.

But the diffusers team has already shipped workarounds that actually matter. They’ve integrated Flash Attention 3 support for Hopper-series GPU owners to claw back some speed. More critically, they’re offering a 4-bit quantization path using bitsandbytes. This compresses both the transformer and text encoder, bringing the entire thing down to a size that fits on a consumer 24GB GPU. The code snippet is up on the repo. You can now run a 32B-parameter model locally that was, minutes ago, confined to datacenter hardware.

The model also introduces a new autoencoder and a better way to handle resolution-dependent timestep schedules. And crucially, it’s built for both image-guided and text-guided generation, accepting multiple reference images as input. That’s a signal BFL is moving beyond just text-to-image into more complex editing and control workflows. The open question now is whether the visual quality jump justifies the compute. If the quantized version holds up, this could be a bigger deal than the raw spec sheet suggests.