Liquid AI's new 230M encoder runs 3.7x faster than ModernBERT on CPU at 8K tokens

Liquid AI just dropped a pair of general-purpose text encoders that solve a specific, grinding problem: running long-context NLP tasks on a CPU without wanting to throw your laptop out the window. The LFM2.5-Encoders come in 230M and 350M parameter sizes, both supporting an 8,192-token context window. The real headline isn’t the model size—it’s the latency curve. While ModernBERT-base chokes on long inputs, taking over 90 seconds for a single forward pass at 8K tokens, the smaller LFM2.5-Encoder-230M does the same job in about 28 seconds. That’s roughly 3.7x faster.

The speed gap widens with length. Liquid AI’s benchmarks show that as input size grows, ModernBERT’s throughput plummets. Their encoders, built on the same LFM2 architecture as last month’s retrieval models, see latency rise gently before tapering off. This matters because the boring-but-critical jobs these models do—intent routing, PII scrubbing, policy linting—run continuously on CPU infrastructure that nobody is upgrading to an H100 cluster anytime soon. The team re-initialized these from their LFM2 decoder backbones, slapped on bidirectional attention and non-causal convolutions, then trained with a 30% masking rate.

On the quality front, the results are scrappy in a good way. The 350M model lands fourth out of 14 on a gauntlet of GLUE, SuperGLUE, and multilingual tasks. The three models ahead of it are all beefier, including one with nearly 10x the parameters. More impressively, the 230M version beats ModernBERT-base and every EuroBERT model tested, despite being smaller than most of them. “Both also score well above our own LFM2.5-Retrievers here,” the team notes, which is exactly the kind of honest, un-flashy comparison you want to see.

Liquid AI also released a set of CPU-only Hugging Face demos that show the encoders doing actual work: zero-shot prompt routing where you define lanes in plain text, token-level policy checking against company rules, multilingual PII detection across 40 categories, and even a spell checker. There’s also a weird bonus demo that runs the encoder as a masked-diffusion chatbot, generating text by iteratively unmasking tokens instead of predicting them left-to-right. It’s a clever reminder that these aren’t just benchmark numbers—they’re building blocks for the unglamorous, high-volume plumbing that keeps production NLP systems from bankrupting you on inference costs.