Tagged: kv-cache

4 articles

Latest

Transformers Run Twice: Why Inference Is Nothing Like Training

DeepSeek's V4 slashes KV cache to 2% of normal size, making million-token agents practical

NVIDIA’s KVPress toolkit slashes 1M-token Llama 3 memory from 330GB to fit on a single GPU

IBM and top schools drop Bamba-9B, a hybrid model with 2.5x faster inference