Liquid AI's DSpark drafters hit 3.18x decoding speed on H100, but MoE lags on Macs

Liquid AI just shipped DSpark, a speculative decoding system that bolts a ~300M-parameter drafter onto three LFM2.5 models to speed up token generation without touching output quality. The headline number is 3.18x faster decoding on an H100 for the 8B-A1B model running MATH500, with the 2.6B and 1.2B-Instruct variants also getting checkpoints. On an M4 Max MacBook Pro, the same family tops out at 2.87x.

The mechanism is familiar to anyone who’s followed speculative decoding: a small model proposes a block of nine candidate tokens, the big model verifies them in one forward pass, and accepted tokens get committed. What’s different here is the drafter’s architecture. It runs a DFlash-style parallel backbone conditioned on the target’s context features, then a rank-256 Markov head restores inter-token dependency so acceptance doesn’t crater at later positions. A confidence-scheduled verifier prunes low-probability suffixes before verification. The drafter has no vocabulary weights of its own — embedding and LM head are tied from the target at load time, which keeps the added memory footprint to around 655 MB in BF16 for the 2.6B model.

Speedup is wildly workload-dependent, and Liquid AI is upfront about it. The 8B-A1B accepts 8.27 of 10 tokens per step on MATH500 but only 4.02 on GSM8K, swinging from 3.18x down to 1.29x on identical hardware. On Apple silicon, the MoE model gains just 1.18x on average — Liquid AI blames the llama.cpp Metal backend’s MoE implementation and the extra weight traffic from verifying multiple experts per step. That’s a real caveat for anyone hoping to run the 8B locally.

The practical sweet spot looks like agentic workloads. Across multi-tool function-calling scenarios, DSpark cuts latency by 57% on average for the 2.6B model — the kind of workload where a model plans, calls a tool, and re-plans several times per user turn. For single-user chat with batch size 1, that’s the scenario speculative decoding was built for. The catch is deployment: checkpoints ship as Safetensors and GGUF with day-one llama.cpp and SGLang support, but no hosted inference provider serves them yet. You’re self-hosting. And under the LFM Open License v1.0, commercial use is free only up to $10M in annual revenue — a ceiling that feels increasingly cramped as these models get genuinely useful.