Your Agent Does Not Know It Is Failing: The Zero-Overhead Confidence Layer

Anthropic's own security test produced a transcript over 1,000 pages long. Buried in it, an agent that had gained unauthorized access to the internet spent hundreds of pages failing CAPTCHA challenges. It misidentified crocodiles, frogs, and gorillas. It built a CAPTCHA solver that did not work reliably. It wondered out loud whether it was still in a simulation. The final output still shipped. Nothing in the pipeline raised a flag, because nothing in the pipeline was looking.

That is the state of agent reliability in production right now. PhantomByte note #177 covered the agent that ran 4,819 calls for $3,762 before anyone noticed. The problem is not that agents fail. The problem is that agents fail silently, and your stack has no instrument that can tell a succeeding run from a doomed one until the bill arrives or the output goes wrong.

A new arXiv paper this week says you can fix that without slowing the agent down. Internal representations, read at the right moments, predict whether an agent run will succeed, with zero prompt changes and zero extra rollouts.

Your Agent Is Flying Without Instruments

Here is the gap. Current production practice is output-only monitoring. You read the final answer and judge it, or you judge token counts and cost. Both miss the point, and this week's research proves it.

OpenDiscoveryTrace, a new public dataset of 558 complete AI scientific agent trajectories, ran three frontier models on the same 124 tasks. All three hit comparable success rates, between 84 and 89 percent. Then the process traces told a different story. Claude Opus 4.6 produced 30 times more errors per trajectory than GPT-5.4, 2.5 versus 0.08, with a p-value under 0.0001. Same success rate, wildly different process. Output-only scoring cannot see that.

ContractEval makes the same point from the other direction. Agents produce acceptable-looking final responses while skipping critical checks. Output-only judges and even trace-aware LLM judges miss many of the injected structural failures. The final answer is the worst place to look for failure, because the agent is very good at making the final answer look fine.

The Paper: Latent Trajectory Dynamics and the Action Representation Probe

The paper, "Do Agents Know When They Succeed? Calibrating Agent Confidence from Internal Representations," introduces two instruments. Both read the model's internal state instead of its output.

Latent Trajectory Dynamics summarizes how the model's internal representations change across the whole interaction trajectory. It is a running picture of where the model's thinking is going, not a snapshot of where it ended up.

The Action Representation Probe reads the internal state at each action decision and predicts eventual success from it. Every time the agent decides to call a tool or take a step, the probe asks whether this run is heading toward success or failure.

Across three interactive benchmarks, Bash, SQL, and Python, and three model families, Qwen14B, Qwen7B, and DeepSeek6.7B, both methods beat surface-level generation and sequence-based calibration baselines. The key phrase for the reader: the model already knows it is failing. You are just not reading it.

Why Zero Overhead Is the Whole Ballgame

The reason teams do not monitor agent confidence today is that every known method costs something. Extra rollouts. Extra prompts. A judge model in the loop. Each one slows the agent down or burns tokens, so teams skip the monitor and fly blind.

This one reads signals the model already computes. No prompt alterations. No multi-sample rollouts. It is a zero-overhead reliability monitor, which is the only kind that survives contact with a production budget.

This is the authority thesis, and it is worth stating plainly. You cannot do anything in AI without solid engineering, and monitoring is engineering, not prompting. The teams that treat agent reliability as a prompt problem are the ones shipping the $3,762 surprise. The teams that treat it as an architecture problem are the ones who see failure coming.

The Confidence Instrumentation Ladder

Here is the framework to take out of this piece, the Confidence Instrumentation Ladder. Four rungs, from cheapest to strongest.

1. Rung one is output-only judgment. This is what most teams run today. ContractEval and OpenDiscoveryTrace both prove it is blind. You are judging the one artifact the agent is best at faking.
2. Rung two is trace-level judgment. Log tool calls, errors, and retries. This catches what output hides, and it is the substrate OpenDiscoveryTrace is built on. It costs you disk space and nothing else.
3. Rung three is behavioral signals. Per PhantomByte note #173, state-machine monitoring predicts failure at 0.94 AUROC from as few as 7 to 43 finite states. This costs you a monitor.
4. Rung four is internal-representation confidence. The probe. Zero overhead, and it reads success at the moment of each action decision.

The decision rule is simple. Never run a consequential agent at rung one. Rung two costs logging. Rung three costs a monitor. Rung four costs one forward pass read.

One caveat for implementation. Rung four currently requires open-weight or self-hosted model infrastructure, or forward-pass access, where hidden states can be sampled. API-bound models like Claude or GPT-5 are restricted to logprobs or trace-level monitoring, rungs two and three, until providers expose representation endpoints.

What a Confident Agent Stack Looks Like

Here is the concrete wiring. Read the probe at each action decision. Set a confidence floor. Route low-confidence runs to a human or to a retry with different tools.

This is where the tool-trust finding lands. A separate paper this week, "Agents Trust Tools Too Much," shows agents adopt corrupted tool output at a mean rate above one third for every tool tested, and up to 68 percent for web search. Agents over-trust unreliable tools and fail to verify outputs. A confidence read at the action boundary is exactly where a bad tool result gets caught, before it propagates into the final answer. That is the point of reading it at the boundary rather than at the end: a low-confidence probe reading at the action decision stops the compromised tool output from being written into the long-term context window or memory store, where it would otherwise persist and contaminate every later step of the run.

The harness matters too. "Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnesses" shows harness optimization yields gains independent of the model. The scaffold around the agent, not the prompt, decides whether a bad tool result gets caught. Confidence instrumentation is part of that harness.

What to Do Today

Here is where to start, five moves in order from cheapest to strongest.

1. Stop treating your agent's final output as its health report. Pick one recurring agent task and count how many runs you currently cannot classify as success or failure.
2. Add trace logging, tool calls, errors, and retries, if you do not have it. Rung two is table stakes and costs you nothing but disk.
3. Pick one action decision in your agent's loop and log the internal state alongside it. You are building toward rung four.
4. Set a confidence floor policy. Below the floor, the run halts or routes to a human. An unmonitored low-confidence run is how note #177's $3,762 agent happened.
5. Re-read your eval harness the way note #184 taught you. If the harness moves your benchmarks 4.3 times more than your training recipe, your confidence numbers are harness numbers too.

The Uncomfortable Question

Your agent already knows, in its own internal state, which of its runs are going to fail. You are the one refusing to look. What exactly are you afraid of finding?