Skills have become the go-to fix for making AI agents more capable without retraining them. You give an agent a compact playbook — steps to follow, checks to run, mistakes to avoid — and it stops fumbling through tasks from scratch. But until now, nobody had actually measured why skills help. A team from Princeton and UC San Diego decided to find out, running 8,135 controlled test comparisons between agents with and without skills on identical tasks.
The headline finding overturns a common assumption. Skills aren’t a knowledge base. They’re a procedural crutch. When an agent with a skill outperformed one without, the advantage came from having a reliable process to follow in 65.7% of cases. Directly supplying missing facts mattered in just 4.5% of cases. In other words, skills steady the agent’s execution — which setup steps to run, which tools to call in what order, which intermediate checks to perform — rather than filling gaps in what it knows.
But the picture isn’t all rosy. In 10% of cases, agents applied an otherwise useful playbook mechanically or in contexts where it didn’t fit. The wrong skill, applied confidently, becomes a new failure mode. And there’s a scaling problem hiding in the retrieval layer. When a skill library grows from 5 to 100 entries, retrieval precision in actual use collapses from 29.6% to 3.3%. Similar-sounding skill descriptions make the matching problem dramatically harder.
This tracks with what we’ve seen in retrieval-augmented generation more broadly: bigger libraries don’t automatically mean better performance. The researchers argue the field needs to stop treating skills as a storage problem and start treating them as a lifecycle — creation, retrieval, application, and refinement. The agents that actually get better won’t be the ones hoarding more playbooks. They’ll be the ones that know which playbook to pull off the shelf, and when to improvise instead.
