Liquid AI is shipping speculative decoding draft models for its LFM2.5 family, and the numbers are substantial: up to 3.18x throughput improvement on an H100 GPU and up to 2.87x on-device. The company open-sourced the DSpark integration upstream in both llama.cpp and SGLang, meaning developers can grab the speedup without waiting on a proprietary serving stack.
DSpark attacks the decode phase’s fundamental bottleneck. Inference is memory-bound — most latency comes from streaming weights out of DRAM, not from compute. Speculative decoding sidesteps this by having a lightweight draft model propose candidate tokens, then letting the target model verify them all in a single forward pass. DSpark combines three tricks: a DFlash-style parallel backbone that produces hidden states for all draft tokens at once, a sequential head that adds inter-token dependency to boost acceptance rates on later positions, and a confidence-scheduled verifier that prunes low-probability suffixes before verification wastes cycles.
The draft models are tiny — roughly 300M parameters each, with a 5-layer attention-only architecture and a block size of 9. Liquid AI trained on a mix covering SFT, chat, code, and function-calling data, running 15 epochs and selecting for highest acceptance rate rather than lowest loss. That’s a telling choice: in speculative decoding, a draft model’s job isn’t to be right on its own, it’s to be right about what the target model would say.
The most interesting number is the 57% average latency reduction on function-calling for the 2.6B model. Agentic workloads are exactly where speculative decoding shines because tool-use calls involve short, structured outputs that benefit enormously from reduced per-token overhead. On an M4 Max MacBook, the 2.6B model now hits around 140 tokens per second — faster than what most proprietary cloud models deliver to a single user. The 8B-A1B MoE model sees more modest gains (18% on-device), a gap Liquid AI attributes to the Metal backend’s handling of expert routing when verifying multiple tokens. The parity story is clean: greedy decoding with DSpark produces byte-identical output to the baseline because rejected draft tokens simply fall back to the target model’s own choices.
