The Compaction Cliff Why Your Agent Forgets at the Worst Time

A new paper out of arXiv names something you have probably seen but could not explain. They call it the Compaction Cliff. It is the sharp performance drop that hits when an agent compresses its memory to fit a context window. The drop is not gradual. It is a cliff.

The numbers are worse than you think. The paper ran 20 production agent configurations and tested Claude Code's own /compact prompt on Sonnet 4.6. One compaction round preserved only 53 percent of safety rules. Five rounds later, that number collapsed to 10 percent. Your agent loses almost all of the hard constraints you gave it after just a handful of compactions. That is not a quality problem. That is structural.

You have hit this. Your agent runs fine for twenty minutes. Then it forgets a constraint you gave it in the first turn. It re-asks a question it already answered. It makes a decision that contradicts something it said five turns ago. You blame the model. You blame the prompt. The real problem is that the memory was compacted, and the compaction threw away the wrong thing.

Last week I covered memory as a security problem in Your Agent's Memory Is the New Attack Surface. That piece was about how attackers poison your agent's state. This is the operational twin. Not how memory gets attacked, but how it degrades on its own. Five papers in this batch attack the problem from five different angles. The compaction cliff is now a named, researched, measurable failure mode. Your agent does not have a memory problem. It has a compaction problem.

THE CLIFF IS NOT GRADUAL

The paper identifies a specific failure pattern. Performance holds steady as context grows. Then the agent hits a compaction event, the moment it must compress to fit a limit, and performance drops sharply. The drop lands at the exact instant of compression, not as a slow slide across ordinary turns. Token decay erodes you turn by turn; compaction logic cuts you off in a single moment. Not slowly. Not gradually. A cliff.

This changes your mental model. Most engineers treat context limits as a soft ceiling. You approach it, things slow down, you deal with it. The data says otherwise. The ceiling is hard. When you hit it and compress, you fall off. The agent does not get slightly worse. It gets dramatically worse on exactly the tasks that depend on early context.

The implication is uncomfortable. Any agent running long enough to trigger compaction is running on degraded memory. The longer it runs, the more compactions it triggers, the worse it gets. There is no plateau. There is only a series of drops.

WHY COMPACTION LOSES THE WRONG THINGS

Standard compaction works by summarizing or truncating older context. The problem is that summarization is lossy in unpredictable ways. The summary keeps the broad strokes. It drops the specifics. But specifics are often what the agent needs later: the exact constraint, the specific error message, the precise parameter value.

The paper frames this as an information preservation problem. It is a sharp one. In a real agent, a safety rule and an episodic log compete for the same tokens. When the budget overflows, both get summarized at the same rate. But only the rule needs exact wording to remain enforceable. The log does not. A compactor that treats them equally destroys what matters.

This is the core insight: the compaction algorithm does not know which details will matter in the future. It optimizes for brevity, not for downstream relevance. So it keeps the summary and drops the constraint. Five turns later, the agent violates the constraint because the summary does not contain it.

This is not a model intelligence problem. It is a memory architecture problem. A smarter model does not fix it. A better memory system does.

THE FIVE FIXES

Today's batch contains five papers that approach the cliff from different directions. Each one addresses a piece of the problem.

MegaMem goes the retrieval route. Instead of compacting, it retrieves. It is a source-resolved dual-view system built for context windows that run into the hundreds of millions of tokens. It separates semantic access from generation evidence, so you search the whole memory but only load the ranked evidence you need. On its own benchmark of over 500,000 enterprise documents, it pushed overall score from 68.22 to 82.26. The agent does not compact. It fetches what it needs when it needs it. That sidesteps the cliff entirely by never compressing in the first place. The tradeoff is retrieval latency and the quality of the index.

ECHO builds a memory plane, not a context window. It is inspired by how the brain encodes episodic and semantic memory, and it separates memory into types with explicit mechanisms to manage each one. The key innovation is auditability. Every memory operation can be inspected and verified. You can see what the agent stored, what it retrieved, and why. That turns memory from a black box into something you can actually diagnose.

HERO personalizes the retrieval. It uses a profile of the user to decide what the agent pulls from memory. Not all context is equally relevant. HERO's insight is that the user's profile is a signal for what matters. If you are a developer working on a specific codebase, the memory system prioritizes context relevant to that work. It retrieves less but retrieves better, which reduces the pressure to compact in the first place.

MemGuard gates what enters memory. It persists verifier signals as lifecycle metadata, so the system knows whether a memory was ever confirmed as correct. Not every tool output, not every failed trajectory, not every misleading observation deserves a permanent home. MemGuard keeps memory clean at the door, which reduces the compaction pressure you would otherwise accumulate later.

Context as an Environment treats the whole thing as a control problem. The paper, which names its system Scroll, frames context as an environment the agent must manage programmatically. It backs the session with an append-only event log and a sandboxed Python kernel. The agent writes code to search and transform its own state, instead of serializing everything into the prompt. When the working view nears its budget, stale spans get evicted but stay recoverable. This is the shift from passive compaction to active memory management.

THE SKILL ROT CONNECTION

Compaction is not just a context window problem. It applies to skills too, and you can name this cousin of the cliff temporal compaction. Context compaction strikes at the instant a budget overflows; skill rot strikes at the instant code updates outdate a stored skill. Repo2Skill-Evo shows that skills extracted from code repositories go stale silently as the underlying code changes. Across 57 real repositories and 105 release transitions, every single transition invalidated part of the skill set. And the frontier agents trying to maintain them only scored between 29.9 and 69.7 percent on a grounded freshness metric.

The skill exists in your agent's library. It looks valid. But the code it was derived from has moved on. The skill is a compacted version of a moving target.

I covered this from the library side in Your Agent's Skill Library Is Rotting at 100 Entries. That piece showed retrieval precision collapsing as the skill count scaled up. Skill rot is the temporal version: even if retrieval works, the skill itself may be stale. Compaction and rot are the same disease. Lossy representations of things that change over time.

THE REAL-WORLD EVIDENCE

The "200 Billion Tokens Later" retrospective is a concrete data point. A developer let agents work for a month decompiling Modern Warfare 2, and the effort burned roughly 200 billion tokens. A month-long agent run is a compaction cliff stress test. The agent must compact repeatedly over a long horizon, and the write-up documents exactly where memory degradation bites. It is a candid look at the scale of compute that sustained agentic work consumes, and at the limits agents hit when their context outgrows the window.

Anthropic's Claude Cowork now adds persistent memory, and the industry is treating it as a feature win. It closes the gap where agents forget context between sessions. Good. But persistent memory without compaction management just delays the cliff. The memory grows until it hits a limit, then compacts, then degrades. The cliff is still there. It is just further away.

THE MEMORY ARCHITECTURE DECISION

Here is the framework I keep coming back to. Call it the Three-Layer Agent Memory model.

Layer one is working memory. That is your current context window. It is fast and it is small, and it is exactly where the cliff lives. Layer two is retrieved memory. That is the MegaMem and HERO style on-demand retrieval. Layer three is persistent memory. That is the ECHO and MemGuard style auditable long-term storage.

The compaction cliff happens when you try to use layer one for everything. You stuff everything into the context window, hit the limit, compress, and fall off the cliff. The fix is to use all three layers. Working memory holds the current task. Retrieved memory fetches relevant context on demand. Persistent memory stores long-term state with auditability.

The decision is not which model to use. It is which memory architecture to build. A model upgrade does not fix the compaction cliff. A memory architecture upgrade does. If you are buying a bigger context window and calling it a day, you are buying a bigger cliff.

WHAT TO DO TODAY

Identify your compaction trigger. Run your agent on a long task. Monitor context length. Note the exact turn where compaction fires. That is where your cliff begins. Measure agent performance before and after that turn. The delta is your compaction cost, and it is a number you can actually defend.

Stop relying on summarization-only compaction. If your agent summarizes old context to fit limits, audit what the summaries drop. Feed the agent a task that depends on an early-context detail. If it fails, you have proven the cliff is real in your system. You do not need a paper to tell you it is there. Prove it yourself.

Implement retrieval-based context expansion. Instead of loading everything into the context window, store context externally and retrieve on demand. Start simple. A vector index over past turns. Retrieve the top five most relevant past contexts for each new turn. That is the MegaMem approach at its simplest, and it is enough to move most of the problem out of the cliff zone.

Gate what enters memory. Not every tool output, not every sub-agent result, not every intermediate step deserves a place in context. Add a filter that scores content relevance before it enters the window. That is the MemGuard principle. Keep memory clean to reduce compaction pressure.

Audit your agent's memory operations. Log what the agent stores, what it retrieves, and what it forgets. ECHO makes auditability a first-class design principle, and you should too. If you cannot inspect what your agent remembers, you cannot diagnose the cliff when it hits.

Test skill freshness. If your agent uses skills derived from code or documentation, check whether the source has changed since the skill was created. Stale skills are silent failures. The agent looks like it knows something. It does not.

THE UNCOMFORTABLE QUESTION

Your agent runs for an hour, forgets what you told it in the first five minutes, and you blame the model. You blame the prompt. You blame the vendor. The paper has a name for what actually happened, and it is your architecture, not your model, that threw it away. When was the last time you looked at what your memory system actually lost?