Google Research has a new trick for making AI agents better over time, and it doesn’t involve touching the model weights at all. WikiSkill pairs an agent with a three-tier memory system: raw execution traces at the bottom, a distilled wiki of failure patterns and winning strategies in the middle, and reusable skill instructions on top. The agent never truly learns continuously — that’s still unsolved — but it writes better instructions for itself after each run and pulls them up next time. The wiki layer only grows; skills can be rolled back if a proposed change flunks validation.
The numbers are substantial. Gemini-3.5-Flash jumps from 49.5 percent to 68.1 percent average across five benchmarks covering math, web search, spreadsheets, document QA, and virtual environment tasks. Qwen-3.6-27B climbs from 39.4 percent to 63.3 percent. Individual gains are even starker: Gemini nearly doubles on LiveMath, going from 33.0 percent to 72.6 percent, and hits 76.6 percent on SpreadSheet, up from 50.5 percent. The work draws on Andrej Karpathy’s ‘LLM Wiki’ concept, which argued for compiling experience into persistent, cumulative knowledge rather than discarding it after each run.
The pattern isn’t uniform. Math and spreadsheets improve most, while long-context document tasks like OfficeQA see far smaller gains. Smaller models struggle to reliably execute evolved multi-step strategies across long contexts and fall back to default behavior. Larger models benefit more, but WikiSkill lets smaller models match the performance of larger ones that don’t use the framework. Skills also transfer between models — sometimes, and sometimes they even outperform what the receiving model built itself. The researchers advise checking transferability case by case rather than assuming it’ll work.
There’s a gating mechanism worth appreciating here: a Skill Proposer suggests changes, a validation set tests them, and if performance doesn’t improve, the skill rolls back. But the wiki keeps the failed attempt documented, so the system learns from what didn’t work. That’s the kind of design choice that separates a research demo from something you could actually deploy. It’s not elegant — it’s arguably a workaround for a fundamental limitation — but the benchmarks suggest the workaround works. Whether it holds up outside controlled benchmarks is the question Google’s next experiments will have to answer.
