Tagged: deepspeed

5 articles

Latest

DeepSpeed ZeRO-2 fits 5X larger batches on DeBERTa-XL, cutting training time 3.5X

BLOOM hits under 1ms per token on 8 A100s, but setup is everything

Hugging Face Optimum cuts model training time by 35% with ONNX Runtime

DeepSpeed ZeRO-3 clashing with Liger GRPO loss? Shape mismatch halts Qwen2.5 finetuning

DeepSpeed was secretly upcasting to FP32 — here's why your FSDP loss diverged