Escaping AI Idea Basins: Why Flushing Chat History Boosts Performance

Microsoft Research ran the same agent on the same task eight times. Same model, same prompt, same tools, same empty starting workspace. Final scores ranged from 68.3 to 100.0. The researchers then gave the runs substantially more compute and the gap stayed wide. The runs refused to converge.

The reason now has a name. In a paper posted October 5, 2026, Ziyang Cai, Christos Ziakas and colleagues mapped every submission the agent produced onto a two-dimensional grid where similar answers sit close together, and solutions from a single run almost always stayed inside one small region. The authors call that region an idea basin. Your agent's chat history is not memory. It is gravity.

The fix reads like heresy. Throw the conversation away and keep the workspace. Their method, Fork-and-Flush, scored a min-max normalized average of 0.78 across 13 long-horizon tasks, against 0.47 for single runs and 0.54 for best-of-4, under an equal compute budget. That is a 66.0 percent relative gain over a single run and 44.4 percent over best-of-4, and it took the highest mean score on 12 of the 13 tasks. If you read our field note Your Agent Swarm Is a Token Fire and concluded that parallelism is a cost trap, this is the version that pays, because nothing crosses between branches except the files.

WHAT AN AGENT IDEA BASIN IS

The eight-run experiment ran on p171, same agent, same tools, same prompt, same workspace, through GitHub Copilot CLI version 1.0.73. Every run plateaued, each at a different level, from near perfect down to more than 30 points below however much compute was added.

The researchers mapped what each run actually explored: deduplicate the graded submissions by content hash, subsample up to 51 artifacts per run, rewrite each as a functional specification so syntax and scaffolding cannot make artifacts from one run look alike for the wrong reasons, then compare artifacts in triplets and fit a two-dimensional map using t-distributed stochastic triplet embedding.

That map agrees with 85 percent of held-out triplet judgments against 50 percent for chance, and each run's submissions cluster in one region and never leave.

For the newcomer, picture a hiker who only ever walks downhill: every step an improvement, every step in the same valley. A hiker camped a mile away stands on a better summit, and the two never meet.

For the engineer, the mechanism: each token of prior context biases the next generation toward a continuation of what already happened, so after a few hundred iterations the transcript is no longer a record of what worked, it is a prior over what comes next, and that prior points inward. The search narrows instead of widening. This is a property of the loop, not the prompt.

WHY MORE CONTEXT MAKES IT WORSE

The uncomfortable part is that context is supposed to be the agent's memory of what worked. The evidence says otherwise.

The same paper reports that this localization survives periodic compaction of the conversation history. Compaction is the tool most teams reach for first, and it is not enough. A summary of a bad line of reasoning is still a bad line of reasoning, and it still steers.

SpireBench, a long-horizon benchmark built on the deck-builder Slay the Spire and posted to arXiv on October 8, 2026, supplies the negative result that makes the basin thesis legible. Across 20 large language models at the game's highest difficulty, the authors find that models do not improve from the memories they write across runs, and that those memories are often wrong and overly cautious. Your transcript is not a knowledge base. It is a ratchet with one direction.

X-AgentRed, posted October 9, 2026, makes the same point from the attack side. It finds harmful tool call paths emerging from cross-service combinations no model has seen, at 1.77 times the harmful path rate of the STAC baseline and 2.99 times the tool coverage. Those paths come from walking a dependency graph, not from recalling history, which is the lesson our incident map in Your Agents Fail at the Seams, Not in the Model delivered.

Our note Your 128K Context Is 6x Slower Than It Needs to Be measured the first bill long context charges you, throughput. This is the second bill, and you cannot pay it off with faster prefill. Variance across runs was 0.26 in the single-run condition, meaning identical code, prompt, and model land on materially different answers depending on luck.

FORK-AND-FLUSH, THE MECHANIC

Here is the pattern, step by step, so you can implement it this week.

1. Fork. At a trigger point, split the agent into K parallel branches, each inheriting a copy of the current workspace files but starting from an empty conversation context. The workspace is the work. The conversation is the steering.
2. Run each branch for a fixed exploration budget. In the paper's configuration K is 4, each branch gets roughly 0.06 of the total budget, and the run uses a forced cadence of three fan-outs. The first launches four agents from empty workspaces and promotes the best to leader. The next two fire at 33 percent and 66 percent of budget, on budget consumption rather than a detected stall.
3. Keep the winner and prune the rest. The highest-scoring branch becomes the new leader and continues serially until the next trigger. Nothing is merged, nothing is averaged, nothing is debated.
4. Flush again from the winner if the budget allows.

Notice what is deliberately missing: no plateau detector. The authors tried a stall-clock trigger and found it rarely fired, because the credit cost of one step varies by orders of magnitude across tasks. A fixed schedule beat it, which makes this a scheduling change and not a model change, and that is why the result belongs on your side of the stack.

The headline numbers on equal compute: 0.78 for Fork-and-Flush, 0.47 for a single run, 0.54 for best-of-4. Normalized standard deviation fell from 0.26 to 0.16. It is not a single-task fluke either, because the method won on 12 of 13 tasks spanning polyomino packing, job-shop scheduling, cryptanalysis, inferring a groundwater source from well readings, compressing a CIFAR classifier under an accuracy constraint, and optimizing a VLIW kernel.

LongHarness, posted October 9, 2026, argues the same from the retrieval side. Its Evidence-Conclusion Coevolution mechanism is model agnostic, and on OfficeQA Pro it lifts GLM-5.3-Flash exact match from 15.8 percent to 60.2 percent on V1 and 4.4 percent to 61.1 percent on V2, purely by managing evidence and conclusions inside the harness.

THE 22 PERCENT CHEAT CODE

The detail most coverage skips. Flushing the context without any branching still improved results. On the 11 tasks where the authors isolated the mechanism, the single-run baseline scored 0.45 and flushing alone scored 0.55, roughly a 22 percent relative gain.

You do not need a parallel runtime to collect a real gain this week. If your agent has been chewing on one task for hours, save the workspace, wipe the conversation, and restate the goal cold. That is the cheapest intervention in this research line, and it contradicts the default assumption that more context is always better.

This ties back to our note Your Evaluation Harness Moves Your Benchmarks 4.3 Times More Than Your Training Recipe. The loop around the model moved these numbers. No new checkpoint was involved and no weights changed. The only state that propagates across branches is the workspace.

WHEN NOT TO FLUSH

Short tasks are unaffected, and this boundary matters most. The basin effect appears in long-horizon runs, from hours to days, where an agent iterates hundreds of times against one objective. It does not appear in a five-minute query or a utility loop that resolves in one or two turns. Flushing a short-context task does not help and actively costs you, because those loops depend on working context for coherence and never accumulate the history a basin needs. If your task finishes inside one context window, leave it alone.

Audit and regulated work carries a compliance cost. If your process requires the full transcript as a record, flushing destroys it, and the gain is a policy problem you price before you schedule it.

Understand what Fork-and-Flush keeps. The workspace files persist across every branch, so the accumulated work is never discarded. Only the conversational gravity is.

The authors name their own limits. The runs are expensive, which capped the task count at 13, and the branches execute independently with no coordination.

One more result matters. On MAX-3-SAT problems across Qwen3.5-4B, 9B, and 27B, run-to-run variance follows an inverted U, peaking when a model attains roughly half of the reference solver's performance. The basin effect is worst exactly at the boundary of a model's ability, and since open research problems sit at that boundary by definition, the authors expect it to persist as models improve.

WHAT TO DO TODAY

1. Pick your longest-running agent task, the one where retries keep producing variations of the same failed answer.
2. Run it three times from scratch tonight and record the score spread. If your runs cluster the way 68.3 to 100.0 did, you have an idea basin.
3. Implement the cheap version first. On plateau, save the workspace, wipe the conversation, and restate the goal cold. Ballpark from the data, up to 22 percent.
4. If you have a parallel runtime, implement the full schedule, four fresh-context branches sharing workspace files, keep the winner, prune the rest.
5. Track normalized standard deviation across runs before and after. A falling spread, 0.26 to 0.16 in the paper, is your proof that the fix worked.

THE UNCOMFORTABLE QUESTION

Your agent's transcript is the thing you have been paying to grow. If throwing it away improves the answer by 66 percent, what exactly have you been storing all this time?