DeepSpeed ZeRO-3 clashing with Liger GRPO loss? Shape mismatch halts Qwen2.5 finetuning

Another day, another obscure shape mismatch tearing through a perfectly good training run. This time, the culprit is a collision between Liger Kernel’s fused GRPO loss and DeepSpeed ZeRO-3 while trying to fine-tune a Qwen2.5-0.5B-Instruct model in bf16 precision.

The stack trace is a pretty clear indictment. The GRPOTrainer.compute_loss method from the TRL library hands off to Liger’s optimized LigerFusedLinearGRPOFunction. Everything goes sideways inside fused_linear_ppo.py during the chunked forward/backward pass. The accumulate_chunk function calls fused_fwd_bwd, and that’s where PyTorch Dynamo throws a fit trying to compile the graph. It’s not a subtle numerical error—it’s a hard tensor shape disagreement, which usually means the model’s partitioned parameters under ZeRO-3 aren’t lining up with the shapes Liger’s kernel expects for its fused operation.

This isn’t entirely surprising if you’ve been tracking the fine print on these integrations. Liger’s fused kernels are built for speed by shoving multiple operations (like the linear layer and the GRPO loss calculation) into a single CUDA kernel. DeepSpeed ZeRO-3, meanwhile, shards parameters, gradients, and optimizer states across GPUs, gathering them on the fly only when needed. Getting the two to agree on what a “full” weight tensor looks like at the exact moment Liger demands it is the tricky part. The Qwen2.5-0.5B model is tiny, so this isn’t a memory issue—it’s purely an interface one.

For practitioners hitting this wall, the immediate debug path is to check whether the Liger kernel is correctly gathering the sharded parameters before its fused operation begins. A temporary workaround would be dropping down to ZeRO-2, where parameters are replicated, to prove the Liger loss works in isolation. The smarter long-term fix likely lies in an updated version of the TRL or Liger integration that explicitly handles the all-gather step ZeRO-3 requires before the fused kernel fires. Keep an eye on the Liger Kernel GitHub—issues like this tend to get patched quickly once the exact parameter sharding pattern is reported.