Kuaishou's KAT-Coder cracks 100K repo tasks by fixing a 16% infrastructure error

Kuaishou’s KwaiKAT team just dropped KAT-Coder-V2.5, and the most interesting part isn’t the model weights—it’s the training pipeline. They built a system that doesn’t just judge code by looking at final test results. Instead, AutoBuilder creates over 100,000 executable repository environments across 12 languages to verify a model’s work. The core insight is brutal: filtering training data by whether a test passes is “misleading.” Some passing code hard-codes answers or bypasses mechanisms entirely, while many failed runs contain brilliant search and repair logic worth learning from.

So the team flipped the script. They generate hints for near-misses, nudging the pass rate for previously impossible tasks to roughly 20%. Then they strip the hints and regenerate a clean, hint-free trajectory. Only samples that pass verification, show no leakage, and stay consistent with the patch are kept. For passing runs, a scoring stage checks for nine distinct qualities, including exploration, recovery behavior, and “honesty.” Harness overfitting is attacked by randomizing tool names, argument conventions, and prompt formats while preserving functionality.

But here’s the real kicker. During V2 training, slow reward curves were blamed on the RL algorithm. An audit found the truth was messier: roughly 16% of trajectories failed because of sandbox infrastructure, not bad policy. Disk usage hit 95%, environment variables were wrong, and a mainstream chat endpoint caused 40% token drift on long rollouts. “Boundary misalignments sometimes emptied observations for ~40 steps and corrupted rewards,” the team noted. Fixing the disk eviction policy, environment variables, and bypassing chat endpoints for direct generation slashed the error rate below 2% and cut training collapses by an order of magnitude.

They paired this with an asymmetric PPO setup and a three-tier reward that scores build success, test collection/execution, and test pass rate. The result is a model trained to operate inside real repositories, not just emit single-turn snippets. The served model is available through StreamLake, and an open-weight Dev variant is on Hugging Face under Apache 2.0. It’s a rare look at how much engineering grunt work sits behind a clean benchmark score.