Video generation has a math problem. Flatten a five-second 720p clip into tokens for a model like Wan2.2-14B, and you get roughly 70,000 spatiotemporal elements. On an RTX 5090, processing attention for those tokens consumes more than 64% of total generation time. Nunchux AI argues this bottleneck is structural, not just hardware-limited. Their new solution, VC-Attention, targets the two biggest inefficiencies in low-bit video Diffusion Transformers (DiTs): value quantization error and the sluggish softmax stage.
Most prior attempts to speed this up, like SageAttention2, smooth queries and keys but ignore values. This leaves 82% of output error stemming from value outliers. Nunchux introduces V-Smooth to fix this. Instead of rotating data, which barely helps, V-Smooth uses online k-means to group value tokens per batch. It subtracts the mean of each 128-token block, quantizes only the residual using 4-bit or 8-bit precision, and adds the mean back later. This simple shift removes up to 36% of block energy after sorting. The overhead is minimal, costing just 3% to 4% of total attention time because grouping runs only during the first quarter of denoising steps.
The second innovation tackles the softmax function, which typically forces data back into high-precision FP32. ExpCast-FP8 bypasses this entirely. Since E4M3 bytes already resemble logarithmic values, the method writes the byte directly from the log-domain score using a single fused multiply-add operation. It avoids the expensive exponential calculation altogether. Benchmarks show this direct path matches the standard FP32 cast on nearly 80% of values, with a measured average error of just 1.6% across 204,800 attention rows.
The performance gains are stark. On NVIDIA’s B200, VC-Attention runs 6.02x faster than SageAttention2. Even on consumer hardware like the RTX 5090, it stays within 5% fidelity loss while matching SageAttention3 speeds. For MiniMax-H3 at 1344x768 resolution, attention is 1.60x faster than BF16 FlashAttention-4, with a slight PSNR bump to 20.2 dB. While Nunchux hasn’t released public kernels yet, their proprietary extension suggests that training-free optimization can finally make high-resolution video generation viable on current silicon. The question now is whether competitors can replicate these gains without Nunchux’s closed-source stack.
