Your Agent's Skill Library Is Rotting at 100 Entries

Princeton and UC San Diego ran more than 8,000 test runs on agent skill libraries and found something that should make you freeze before you add one more skill to your agent's toolbox. When the library grows from 5 entries to 100, retrieval precision collapses from 29.6% to 3.3%. Your agent stops finding the right skill. It starts grabbing whatever sits at the top of the embedding pile.

You built a skill library because everyone said skills make agents better. They do. The study confirms it. Skills help through set procedures, not factual knowledge. Procedural grounding accounted for 65.7% of the cases where a skill-equipped agent beat one without it. Directly supplying knowledge helped in just 4.5% of cases. But nobody told you what happens when the library gets big. It rots.

We covered this gap from the architecture side in #122, Your Agent Is a Monolith. Give It a Shepherd. We covered the tool-description tax in #128. Both assumed your agent could find the right tool. It cannot. Not past a certain size. The retrieval layer is the bottleneck now, and almost nobody is building for it.

The Procedure Advantage

The Princeton and UC San Diego study isolates what skills actually do for agents. They do not inject knowledge. They inject procedure. 65.7% of the performance lift comes from the agent following a set playbook, not from facts the skill contains. Only 4.5% of the lift comes from directly supplied knowledge.

This reframes the skill question. You are not building an encyclopedia. You are building a library of playbooks. Each playbook is a sequence the agent executes. The value is in the procedure, not the payload. When you think about it that way, the retrieval problem gets sharper. Your agent does not need to find the skill with the right facts. It needs to find the skill with the right steps. Semantic similarity search, which is what most skill retrieval systems use, was never designed for that.

The Collapse

Here is the number that matters. At 5 skills in the library, retrieval precision is 29.6%. That is already not great. You have roughly a 1 in 3 chance of pulling the right skill. At 100 skills, precision drops to 3.3%. That is noise. Your agent is rolling dice.

The implication is that adding skills to your agent library is not linear improvement. It is a curve that bends downward. Past a certain point, each new skill you add makes it harder for the agent to find the right one. The library becomes its own source of failure. The study calls this out directly. In 10% of cases, the agent applied a useful playbook mechanically or in ways that did not fit the task. A wrong skill, executed faithfully, is worse than no skill at all.

Why Semantic Search Fails Here

Most agent frameworks retrieve skills by embedding similarity. The query goes in, the nearest skills by cosine distance come out. This works at 5 skills because the space is small enough that even a rough match lands close. At 100 skills, the embedding space gets crowded. Semantically similar skills cluster together. The retrieval system cannot distinguish between "deploy to staging" and "deploy to production" when both contain the word "deploy" and both sit in the same neighborhood.

The "Right Family, Wrong Skill" paper on arXiv formalizes this. A retrieval system pulls a skill from the right semantic family but the wrong specific skill. The agent gets a skill that looks correct, feels correct, and executes the wrong procedure. Because skills carry executable instructions and scripts, a wrong-skill retrieval is not a cosmetic error. It is an agent acting on a procedure that does not fit the task.

The Optimization Framework

A separate paper, "Optimal Skill Selection for LLM Agents with Provable Bicriteria Guarantees," reframes the whole problem. Skill selection is not a search problem. It is an optimization problem. You choose a skill set under a hard token budget to maximize benefit minus context penalty. Current agents score skills independently by semantic relevance and pack the top-k. No quality guarantee. No cost awareness. Redundant skills waste context tokens, and wasted context tokens degrade performance.

The paper's Best Prefix Selection algorithm, BPS, reaches 0.73 measured task success on a contamination-controlled BigCodeBench variant. Released skill routers and text retrievers scored 0.20 to 0.52. That is roughly a 3.5x gap between treating skill selection as optimization and treating it as similarity search.

The framework to remember is this. Every skill you load costs context tokens. Every skill you load competes with every other skill for retrieval precision. The question is not "how many skills do I have." The question is "what is the optimal set of skills for this task under this token budget." That is a different engineering problem than "add another skill to the library."

The Malicious Layer

The same expansion that kills retrieval precision opens a security surface. MaliciousSkillBench, another arXiv paper, assembled a benchmark of 9,740 skills: 7,505 malicious and 2,235 benign. Learned detectors hit 0.882 to 0.932 F1 on in-distribution detection. Under source-disjoint evaluation, they dropped to 0.653 to 0.665. Current detectors do not generalize across sources.

The implication for you is direct. As your skill library grows, you cannot manually audit every entry. You are trusting third-party skills, community skills, or auto-generated skills. The detection tools that exist today cannot reliably catch malicious skills when they come from a different source than the training data. A 100-entry library is not just retrieval-degraded. It is a distribution channel for malicious behavior if you are not curating the source.

The Lifecycle Model

The Princeton and UC San Diego researchers argue that skill use should be treated as a lifecycle. Create, retrieve, apply. The industry obsesses over the create stage. Everyone publishes skill frameworks, skill marketplaces, skill generators. Almost nobody builds for retrieval and application. That is where the failure lives.

The lifecycle model gives you a diagnostic. If your agent is underperforming, do not add more skills. Audit the retrieval layer. Measure precision at your current library size. If it is below 10%, your problem is not "not enough skills." Your problem is "too many skills, bad retrieval."

This connects to #160, Your Cost Curve Just Broke. Harness Efficiency Now Decides Your AI Bill. The harness, not the model, is where you win or lose. Nvidia research showed that a custom harness lifted Claude Opus 5 from 30% to 100% on ARC-AGI-3, while the model alone scored 30%. Skill retrieval is part of the harness. A library of 200 skills with 3.3% retrieval precision is a harness failure, not a capability failure. That is the argument from #124, Your Agent's Harness Is Your Real Model, made concrete.

What to Do Today

1. Measure your retrieval precision. Take 50 tasks your agent handles. For each, check whether the correct skill was retrieved. If your precision is below 15%, your library is too big for your retrieval system.
2. Cap your active skill set. Start at 10. The study shows 5 skills gives 29.6% precision. 10 is probably your sweet spot before the decay curve steepens. You can keep more skills in cold storage, but only load what the task needs.
3. Switch from top-k similarity to cost-aware selection. Score each candidate skill by relevance and token cost. Drop skills that are redundant with ones already selected. The BPS paper shows this approach hits 0.73 success versus 0.20 for naive retrieval.
4. Audit your skill sources. MaliciousSkillBench shows detectors fail on cross-source evaluation. If you are pulling skills from multiple frameworks or community repos, you have a gap. Curate your sources or build source-specific validation.
5. Treat skills as playbooks, not encyclopedias. 65.7% of the value is procedural. Write skills as step sequences, not fact repositories. A skill that says "run these four steps in order" beats a skill that says "here is what you need to know about deployment."

The Uncomfortable Question

You have 200 skills in your agent library. You added them because more capabilities means a better agent. But the data says your agent can only find the right one 3.3% of the time. You are not running a skill library. You are running a slot machine. When are you going to measure what your agent actually retrieves, instead of counting what you gave it?

The Skill Retrieval Decay Curve is real. Skills help through procedure, not knowledge. Past 100 entries, your retrieval is noise. The fix is not another skill. It is an optimization problem with a token budget.