Your Reasoning Budget Is Wasted: A Frozen Table Ranks 3rd of 130

A forecasting system just placed third out of 130 entries on GIFT-Eval, the open benchmark for time-series forecasting. The two systems ranked above it are agentic, meaning multi-step pipelines that use agents or language models to reason about, generate, or select forecasts.

The system in third place runs no agent and no language model.

Its entire decision layer is a table. That table was computed once on the training split and then frozen. Every decision in it was taken on a backtest carved out of data the system was allowed to see, and then it stopped moving. Its mean MASE rank, the scoring position the benchmark assigns across every dataset and horizon it covers, is 19.4 as of September 14, 2026. The best single base model served on its own reaches 33.8. The selection tournament served on every configuration reaches 38.0.

Read the gap between those three numbers again. The frozen router beats the best single base model by 14.4 rank positions and beats the tournament by 18.6. That is the entire value of routing, isolated and measured. The model did not get smarter. The decision about which model to use got better, and it got frozen.

Position 3 of 130 is two rank positions below the top agentic system on the board, which is the entire accuracy distance between a lookup and an agent on this benchmark. Set that against what the two approaches cost to run. The routing decision in TW3Cast requires zero inference tokens at decision time, because it is a table lookup rather than a model call. The agentic systems above it spend a multi-step reasoning loop on the same decision, on every request, and land at most two positions higher. Near-identical rank, and the decision cost is not comparable, it is a categorical difference: a table lookup bills nothing per request, a reasoning loop bills on every call.

We argued before that your router is the moat, not your model, and in "The 93% Problem: AI Reasoning Waste" we put a number on how much reasoning spend goes to decisions that did not need it. That claim now has evidence behind it. This piece is the proof.

And before you file this under forecasting and move on, look at what actually got replaced. Nothing about the mechanism is specific to time series. A table that maps a known input signature to a pre-chosen handler is the same object whether the handler forecasts a series or answers a request. Text routing, tool selection, intent classification, multi-agent orchestration, and model tier selection are all the same shape of decision: given a request, which handler runs. TW3Cast proves that decision can be computed once from your own history, frozen, and audited, and that the frozen version holds or beats the reasoning version. If your pipeline sends a request to a router, a classifier, or a tool chooser and pays a language model to make that call on every request, you are running the expensive version of a decision that has already been shown to work as a lookup.

The uncomfortable version is this. You are burning reasoning tokens on decisions a frozen lookup already makes better. And you cannot audit a prompt. You can audit a table.

THE SYSTEM, AND WHY A FROZEN TABLE BEATS YOUR AGENT

The system is TW3Cast, from Nathan Thierry and Andre-Louis Rochet. It is not a model you can rent. It is a routing layer, and the routing layer is the product.

Here is the shape of it. GIFT-Eval breaks forecasting into 97 configurations, each one a dataset, a frequency, and a horizon. For every one of those 97 configurations, TW3Cast serves exactly one of four modes.

1. The first mode is a specialist, which is a LoRA or a full fine-tune of a public foundation model, Chronos-2, TiRex, or Toto, trained on data cleaned and enriched by explicit rules.
2. The second is a quantile blend that contains a specialist.
3. The third is a blend of base models.
4. The fourth is a selection tournament played on a backtest carved from the training split.

A specialist gets admitted the moment it beats the tournament on that backtest. A specialist costs a few megabytes and minutes of GPU time to evaluate. A failed candidate changes nothing anywhere else in the table.

Sit with that design for a second. A new expert is a candidate row, not a redeployment. You test it against the current occupant on your own held-out data, and if it loses, you delete it and move on. Nothing downstream breaks because nothing downstream was ever asked.

The cost and latency story is the part your finance team cares about. The two systems above TW3Cast are agentic, and the accuracy gap between a frozen table and an agent is two rank positions out of 130. The gap in cost, latency, and auditability is enormous, and in the case of the routing decision itself the cost of the table is zero tokens per request against a full reasoning loop per request. You are not choosing between a fast wrong answer and a slow right one. You are choosing between a lookup and a reasoning loop that lands two positions higher at most, and only one of those two can be explained to an auditor.

There is a companion finding worth stacking next to this. A separate paper, "When Should Forecasting Agents Reason? Behavioral Stress Tests for Reliability Routing," studies when an agent should retrieve, when it should reason, when it should defer to a market prior, and when it should use a historical analog. Its conclusion is blunt: mechanism choice is source-dependent. Structured analogs dominate for some data-generating processes, while market and crowd-style baselines win on others. More reasoning is not always better.

That paper introduces ReliabilityRoute, a rule fitted once on 2024 data. It closely matches a hand-built taxonomy without hard-coding any source names, and a walk-forward version that refits its thresholds from previously resolved vintages posts the best mean Brier score among the deterministic systems across 16 later LLM vintages.

The authors are honest that the gain is modest and that plain search baselines stay competitive. That honesty is exactly why the finding matters. The decision rule should be chosen per data-generating process, not per product roadmap. Reasoning is a mechanism you deploy when the evidence calls for it, not a default you pay for on every call.

THE THREE GUARDS

This is the part you can steal today. TW3Cast protects its own selection with three guard mechanisms, and every one of them exists because the naïve version of the table lies to you.

1. Guard one is a dual accuracy-and-calibration criterion. Winning on point accuracy alone is not enough to hold a seat in the table. A candidate that is right about the value but overconfident about it fails the gate, because a wrong-confidence model poisons everything downstream of it.
2. Guard two is an asymmetric margin against candidates that saw the series during training. Contamination gets a handicap, not the benefit of the doubt. If a model had the answer in front of it during training, it has to clear a higher bar to be believed.
3. Guard three is conservative per-window gates. When the table is not sure, it falls back instead of guessing. Uncertainty routes to the safe option rather than to the confident-looking one.

Why this matters to you is not academic. These three guards are the difference between a routing layer you can ship inside a regulated pipeline and one you have to apologize for six months later. Any reader can adopt all three as a checklist tomorrow morning. Most routing layers in production right now have zero of the three.

WHY THE GAINS LIVE OUTSIDE THE CHECKPOINT

One paper is an anecdote. Four papers on the same pattern is a direction, and today the pattern shows up everywhere in the literature. The four results below do not look alike on the surface, and that is the point. A routing table, a self-improving harness, a speculative drafter, and a reward function are all the same kind of object: an engineered, versioned artifact that sits outside the weights and does the work the checkpoint cannot do for itself. Read them as four instances of one thesis.

Start with the closest match to TW3Cast. Pistis, a 27B and 9B multimodal model family, ships Pistis-Auto-Harnessing. That method improves the agent's inference harness iteratively and raises measured performance without updating a single model parameter and without increasing the interaction budget. The harness is the scaffolding that wraps the model: the prompts, the tool-calling sequence, the retry logic, the control flow. Pistis-Auto-Harnessing treats that scaffolding as the thing to optimize, and the measured improvement lands with the weights held fixed. This is the TW3Cast mechanism wearing different clothes. The intelligence that moved the number was not inside the checkpoint, and the artifact that carries it is the harness, which you own.

The second instance changes the modality and keeps the thesis. Liquid AI's LFM2.5-VL-DSpark adds a 280 million parameter vision drafter on top of a 3B target, an 8.9 percent increase in deployed parameter count. That drafter buys up to 3.13x decode speedup on device and 2.66x on an H100. It is a separate, versioned artifact, released in its own right, with day-one support in llama.cpp, MLX-VLM, and SGLang. The target model does not change. Same pattern as the harness: a self-hosted artifact that sits beside the model, is versioned independently, and delivers the gain the checkpoint did not.

The third instance goes deeper than infrastructure and still holds. Researchers trained Qwen3.5-0.8B with reinforcement learning from verifiable rewards on a search task and varied only the reward shape. The best run reached 0.352 average exact match against a 0.092 untrained floor, a 3.8-fold gain. The industry default, the exact-match-only reward used by Search-R1, was the worst of the three shapes at every seed at the matched training horizon. It was worst even on exact match, the metric it directly optimizes.

Reward shape is not a hyperparameter you inherit from a large-model recipe. It is a design decision, and getting it wrong at small scale does not degrade your result, it inverts it. Notice where the leverage sat in this study. The model architecture was fixed. The parameter count was fixed. The training data was fixed. The only thing the researchers moved was the definition of the objective, and that single design choice swung exact match from the worst result in the study to the best. The reward definition is an artifact. It is specifiable, it is versionable, and it is yours.

Line all four up and the pattern stops being a coincidence. A routing table you froze. A harness you optimized. A drafter you shipped alongside the model. A reward definition you designed. None of them is a checkpoint, all of them are engineering artifacts you control, and in every case that artifact delivered a measurable gain the rented weights did not. This is where the intelligence in the system is actually living, and it is increasingly outside the model.

Stack all of this together and the procurement argument writes itself. The durable operational asset is a versioned artifact you own and can audit. The table. The drafter. The harness. The reward definition. Not the checkpoint you rent, which changes under you, cannot be diffed, and cannot be explained to anyone who asks why the number moved.

YOUR ROUTE TABLE BUILD ORDER

Five steps. Do them in this order.

1. Carve a backtest out of your training split. Never score selection candidates on data they saw. This is the foundation, and if you skip it every number after it is fiction.
2. Enumerate your modes. Cheapest capable model, blend, specialist, tournament. Cap it at four. More modes means more rows to maintain and more chances for a contaminated candidate to slip through.
3. Apply the three guards. Dual accuracy-and-calibration, asymmetric contamination margin, per-window fallback. All three, or you built a table that flatters itself.
4. Freeze it. Pin the model revisions. Write the regeneration script, because the release is only trustworthy if one command reproduces every number in it.
5. Version it like code. When the table changes, you can diff it and see exactly which configurations moved and why. When a prompt changes, you cannot.

WHAT TO DO TODAY

1. Count how many of your agent calls are commodity decisions, meaning routing, classification, and format choice, and price what a frozen table would save you on those calls.
2. Carve one backtest from your training split and score your current router against a hard-coded table. Compare cost, latency, and accuracy on the same configurations.
3. Run the dual accuracy-and-calibration check on your top model. If it wins on accuracy and loses on calibration, you just found your first guard.
4. Pin your model revisions and write the regeneration script before your next model swap does it for you, badly, and silently.
5. Read the TW3Cast release. The routing table, the expert index, the pinned base-model revisions, and the regeneration script are all published. Steal the structure.

THE UNCOMFORTABLE QUESTION

Your agent reasons through every decision because reasoning looks like intelligence on an invoice. A table just made the same decision faster, cheaper, and in a form an auditor can read. So name the thing you are still paying for, and then explain why you cannot replace it with a lookup.