Tagged: speculative-decoding

11 articles

Latest

Hugging Face Slashes LLM Latency by Up to 10x With Assisted Generation

Liquid AI's DSpark drafters hit 3.18x decoding speed on H100, but MoE lags on Macs

Liquid AI's DSpark Draft Models Hit 3.2x Faster Inference on H100

Whisper transcribes 2x faster with speculative decoding, zero accuracy loss

Tencent open-sources AngelSpec to fix why one-size-fits-all draft models fail

Intel Prunes Qwen3 Draft Model to Hit 1.4x Speedup on Local AI Agents

Meta's LayerSkip eliminates the draft model, cutting LLM memory use by up to 50%

Intel and Hugging Face crack 1.5x-2x faster LLM decoding across any model family

Hugging Face squeezes 52% faster LLM inference by guessing when a draft model is wrong

Google drops Gemma 2 2B: a 2.6B on-device model that punches above its weight

Hugging Face packs Whisper, Pyannote diarization, and speculative decoding into one endpoint