Your AGENTS.md Is Hope at Scale: Only 3.7 Percent of Agent Rules Name the Check That Enforces Them

Someone scripted a sweep of 1,866 of the most popular active GitHub projects and read the agent instruction files inside them. 627 projects, 34 percent, ship at least one. AGENTS.md is the dominant format. Half of the main files run past 100 lines, and the median is 106 lines, roughly 850 words.

Then comes the number that ends the argument. Across two classified samples, 431 lines read as real rules, and 16 of them named the check that enforces them. That is 3.7 percent. You wrote a hundred lines of policy for your coding agent. Ninety-six of those lines are the agent remembering to be good.

We said this in Your Agents Ship Faster Than Trust. The trust layer is the part nobody builds. This is the receipt, counted project by project.

THE STUDY THAT COUNTED YOUR HOPES

An instruction file is a plain text file, usually called AGENTS.md, that a coding agent reads as standing orders before it touches your repository.

Rakesh Menon wrote the analysis and published it on 15 October 2026 at secondtimeisabug.com, the site for his book "The Second Time Is a Bug." He took the GitHub Ranking lists of most-starred repositories, top 100 overall and top 100 in 24 languages: 2,424 projects. 1,867 had a commit in 2026, and 1,866 could be read.

627 of the 1,866, or 34 percent, ship at least one agent instruction file. AGENTS.md is the common ground at 489 of those projects, 78 percent. CLAUDE.md appears in 328 and GitHub Copilot's instructions in 123. TypeScript projects ship one at 73 percent. Perl is 4 percent.

Length is where the picture turns ugly. 302 of 582 main files, 52 percent, run past 100 lines. 87 run past 300. The longest is 1,048 lines.

Then the classification. The script pulled 300 lines that look like rules, and a separate reviewer found 257 that actually issue an instruction. Of those, 157 could be checked by a script, a linter, a test, or a CI job at least in part. 64 could be checked completely, 93 partly. Only 8 of the 157 named the check. An earlier sample of 200 lines came out the same way, 8 of 174. Put both together and you get 16 of 431.

Adoption is real. Enforcement is not. Teams are writing policy at scale and wiring up almost none of it.

Menon's line is the one to keep: an unenforced rule is a hope, and most agent configuration today is hope at scale.

THE 20 PERCENT TAX ON WISHES

The study also points at harder evidence that the file is not free. A team at ETH Zurich led by Thibaud Gloaguen evaluated repository context files across SWE-bench tasks and across real repositories that ship developer-committed files. Their paper is arXiv 2602.11988, first posted on 12 February 2026 and revised on 29 September 2026.

Their finding is blunt. Context files do not generally improve task success rates, and they increase inference cost by over 20 percent on average. That holds across models, across coding agents, and for both model-generated and developer-written files. Instructions are followed well. Repository overviews, the most popular thing in your file, are the part that does not help.

Menon's post renders that as "up to 19 percent." I am using the paper's own number.

Now do the math on your own file. 106 lines is about 1,600 tokens, and it rides in the context of every call the agent makes. Every turn pays for it, cached or not. It sits alongside the code, the tool results, and the failing test, competing for the same attention budget. A double-digit cost premium for marginal gains is not a feature. It is a tax you chose.

There is a counterpoint, and it explains what the good files do differently. Menon cites a study of 124 pull requests where an AGENTS.md cut the median time Codex took by about 29 percent. Both results are true at once. A short file of commands saves time. A long file of preferences spends it.

THE RULE TAXONOMY: THREE CLASSES

The framework takes ten minutes to apply to a file of any size. Sort every line into one of three classes.

Class one, enforced. The rule names its check, the check runs outside the model, and a failure blocks the merge. The check is a linter rule, a test, a CI job, a pre-commit hook, or a permission boundary the agent cannot cross.

Class two, enforceable. A script could catch a violation, but nobody wrote the script. Per this study, that is three in five rules, most of them one small piece of code away from being real.

Class three, aspiration. "Keep the code clean." "Prefer simple solutions." "Be careful with edge cases." Prose that no machine can evaluate and no reviewer can consistently catch. It lives in the file because it felt responsible to write it. It is the same failure as generated code nobody reviews, which I covered in Why AI-Generated Code Is Silently Destroying Your Architecture, moved up one level into the rules meant to prevent it.

Here are three class two rules from the sample, each one step from being real. "Don't use empty destructuring patterns" is one lint rule. "File names should be lower case with words separated by hyphens" is a short script. "Do not add Co-Authored-By: Claude or any Claude attribution to commit messages" is a commit hook, and it is a rule a reviewer will miss at 11pm.

Homebrew shows what class one looks like. Its AGENTS.md is 72 lines, it opens with the one command to run before asking for input, and its rules name their enforcement, including "Never use T.must (enforced by Sorbet/ForbidTMust)." Its commit style rule explains that a GitHub Actions workflow rejects the wrong format.

Kubernetes keeps its file at 38 lines. systemd points the agent at four documents. Both are maps. Neither is a manifesto.

WHY THE MODEL WILL NOT SAVE YOU

Microsoft published the clearest statement of what an instruction file cannot do six days after Menon's post.

Nadim Abdo, Corporate Vice President of Identity and Network Access at Microsoft, wrote on the Microsoft Security Blog on 19 October 2026 that agents belong inside the insider risk model. The reason is not that agents are malicious. It is that the access which makes them useful is the same access that lets them err, be manipulated, or be compromised. His line on the mechanism is the one to save: "An agent that operates as intended when a task begins may respond differently as context or inputs change."

Satya Nadella's framing, which the post quotes, is that the supply of intelligence must be separated from the authority over it, so a model should never be its own security authority. His sharper version: "The most trustworthy agentic system will not be the one with the model we trust most. It will be the one designed that enables us to trust the model least."

Microsoft's architectural advice is the same one I am giving you about AGENTS.md: surround non-deterministic intelligence with deterministic controls, independent evidence, and containment boundaries. Microsoft now inventories 60 types of local agents, Claude Code, OpenAI Codex and GitHub Copilot among them.

Read that against the 3.7 percent. Your instruction file does not bind the model. It persuades it, sometimes, and the persuasion degrades as the context fills and the plan evolves. The model has no persistent conscience. The only thing that binds is a check that fails loudly outside the model, where the model cannot talk its way past it.

THE 5 PERCENT RULE

So here is the decision rule, and it is deliberately harsh. Any line in your instruction file that cannot name its check gets deleted, or it gets the check written. There is no third option.

The target is fewer than 5 percent of rules surviving as pure aspiration. Not zero, because a few judgment calls are genuinely worth stating for a human reader, such as when to escalate instead of guessing. But under 5 percent, so the file reads as a specification and not a wish list.

Deletion is the default, not the failure. The study says most of your file could be checked and does not say how, which means the file is playing policy and reminder at the same time. Dead weight costs tokens on every call and dilutes the lines that matter. A 12-line file with 12 checks beats a 100-line file with none, and it beats a 100-line file with 8, which is about where the best files here land.

Anyone can claim the 5 Percent Rule. Very few teams can produce a gate that proves it.

WHAT TO DO TODAY

1. Open your AGENTS.md and mark every line E, C, or A. Enforced, could be enforced, aspiration.
2. Count the A lines. That number is your exposure, not your policy.
3. Pick the three C lines closest to a lint rule you already run and wire them into CI this week. Three is a real number. It is not a rewrite.
4. Delete the A lines or rewrite them as checks. Do not negotiate with prose.
5. Add one CI gate that fails the build when an instruction file references no check, so the file cannot rot back into a wish list six months from now.
6. Re-measure cost after the trim. If your token bill does not drop, your file was smaller than you thought.

THE UNCOMFORTABLE QUESTION

You would never ship a security policy that nobody audits. You would never approve a permission model that runs on the honor system. So why does your agent policy file get a pass?

Because you assume the model is reading it like a person. It is not. It is reading it like a token stream, with a fixed attention budget, no memory from yesterday, and no consequence for ignoring line 74.

Who is enforcing your rules, your pipeline or your hope?