Cactus Compute just shipped Needle 2, a 45M-parameter tool-calling model that fits inside a single 14MB binary and runs an entire session on about 28MB of RAM. No runtime to install, no model download at inference time. The whole thing is sealed inside the company’s C++ engine, with weights trained and deployed at CQ2-bit — a 2-bit quantization scheme baked in from pretraining, not slapped on afterward. That distinction matters: the deployed model is the trained model, not a degraded approximation of one.
On a Raspberry Pi 5, Needle 2 cranks out 500 tokens per second. Meta Quest 3S and Apple Vision Pro manage 400 to 1,500, while sub-$200 phones land between 300 and 700. Those numbers come from a design bet that’s refreshingly narrow. The Cactus team argues that mapping a messy sentence onto a typed function signature requires zero world knowledge and zero open-ended prose. So they built a Simple Attention Network — 27 layers, 512 wide — that swaps the standard FFN for a Hadamard MLP, keeps GQA attention, and adds engram key-value memory from hashed n-gram tables. The model spends 70 MFLOPs per token. For comparison, Liquid’s LFM2.5-230M burns 460 and Apple’s Foundation Model sits near 6,000.
The engineering gets interesting at the engine level. Weights never decompress into RAM — 2-bit codes expand inside vector registers and fuse into integer dot products, so the arithmetic path stays int8 throughout. A byte-level grammar compiled from your JSON schemas constrains every emitted token, which lets the engine skip up to 98% of the vocabulary projection on structural tokens. Attention uses a 256-token sliding window with the system turn and tool declarations pinned as KV sinks, meaning memory stays flat regardless of how long a conversation runs. Declare more than five tools and a contrastive retrieval head scores each schema per turn, admitting only the top five. Unselected tools are unreachable, not just unlikely.
On evaluation, Needle 2 leads both Seal-Tools splits — 32.6 in-domain and 28.7 out-of-domain using ordered strict exact match — and posts 98.3% function-name accuracy on Mobile Actions. It trails on BFCL v4 at 42.6 overall, which Cactus attributes to corpus distribution: their training data skews toward consumer device actions, not general enterprise APIs. Every response carries a confidence score combining a calibrated head with decoding probability, and off-topic requests return an empty call. That’s the real product decision here: edge-to-cloud escalation stops being a vibe and becomes a threshold you configure. Pebble already runs Needle locally in the Index 01 app for offline voice actions, which suggests the wearables crowd is paying attention.
