Tagged: distributed-training

18 articles

Latest

Hugging Face brings PyTorch training to Google Cloud TPUs with zero interface changes

Ray Cuts RAG Document Retrieval Time in Half for Hugging Face Users

Hugging Face and AWS team up to cut BART training time with one line of code

Hugging Face's Accelerate cuts distributed training to five lines of code

40 Volunteers Just Pretrained a Bengali BERT Over the Internet

Fine-tune BERT 4x faster on Intel CPUs, no GPU cluster required

PyTorch FSDP trains GPT-2 XL on 2 GPUs where DDP crashes with OOM

DeepSpeed ZeRO-2 fits 5X larger batches on DeBERTa-XL, cutting training time 3.5X

Distributed training still trips up ML teams — here's the fix

Sapphire Rapids CPUs cut deep learning costs by up to 90% with AMX acceleration

Cursor drops MoK megakernel: 2.37x faster MoE training, but it demands a $2M rack

Moonshot AI drops MoonEP, claims 2.5x scaling boost for trillion-parameter models

OpenAI Five crushed Dota 2 champs using 2M frames per second

Async RL's dirty secret: 98% of weights don't change between steps, and TRL now exploits that

GPU Math: Why AWS's Latest Instances Won't Fix Your Memory Bottleneck

Hugging Face's New Multi-GPU Recipe Cuts Through Training Complexity

Hugging Face's Accelerate hits 1.0 after 3.5 years, baking in FP8 and bigger model support

DeepSpeed was secretly upcasting to FP32 — here's why your FSDP loss diverged