Your 4-Bit Model Just Beat Its Full-Precision Original. The Compress-Then-Heal Pipeline Is Broken.

A paper dropped this week with a result that should not be possible. Researchers took a GPT-OSS 120B model, compressed it to 60B parameters, quantized the weights to 4-bit MXFP4, and then applied a recovery recipe called Quantization-Aware Healing, QAH. The compressed, quantized model beat its own full-precision bfloat16 version on 7 of 9 benchmarks.

You have been told the deal your whole career: cut the model, lose capability. Every serving guide treats compression and quantization as taxes you pay for cheaper inference. This result says the tax is negotiable, and the negotiating table is the healing step.

The model, released open-weight as Hypernova-60B, is smaller, cheaper to run, and on most tasks more accurate than the full-precision checkpoint it was quantized from. That should not happen, and the fact that it does is the most useful engineering story of the week.

WHAT COMPRESSION AND QUANTIZATION ACTUALLY COST YOU

Structural compression cuts parameters out of the network itself. Fewer layers, heads, or neurons, so the model holds the same job with less machinery. Quantization shrinks the numbers that represent each weight, dropping from bfloat16 down to 4-bit MXFP4, which uses roughly a quarter of the memory. That is the simple version for anyone new to it.

The real cost shows up when you run the standard pipeline. The usual recipe cuts parameters first, then quantizes the surviving weights, then tries to heal the damage. And healing, here is the catch, was built for one assumption only: that quantization was the only wound. The recipes everyone reaches for, quantization-aware training above all, were designed to recover what quantizing lost. On a model that was also structurally compressed, they leave reasoning, math, and code capacity on the floor.

WHY STANDARD HEALING FAILS ON COMPRESSED MODELS

Here is the gap the paper names. Healing recipes recover what quantization noise destroyed. They do not recover what a scalpel already removed. A structurally compressed model has fewer parameters holding the same task, so quantization noise lands on a network with less redundancy to absorb it. Two damage sources, one repair kit built for one of them.

Quantization-aware distillation has the same blind spot from the other side. It distills a frozen full-precision teacher into the quantized student, and that works when the only change was bits. But a structurally compressed model has no independently trained full-precision version of its smaller architecture. The only teacher available is the recovered bfloat16 checkpoint, which is itself a distilled approximation of the original. Distill from that and you anchor the student to a degraded target. The ceiling is baked in before you start.

That is the engineering lesson, and it generalizes past this paper. Your recovery tooling has to know what your damage model is. If you are fixing the wrong failure, you are not healing, you are just re-training into the same cliff.

THE QAH RESULT, IN NUMBERS

Now the citable block. A GPT-OSS 120B model compressed to 60B parameters, quantized to MXFP4, then healed with QAH. Against the full-precision bfloat16 checkpoint of that same 60B architecture, the 4-bit model wins on 7 of 9 benchmarks. MMLU-Pro and SciCode are the only two it trails, by less than a point and a half each.

The gains land exactly where compression usually does the most damage. Long-context reasoning is up 7.4 points on AA-LCR. Math is up 5.6 on AIME 2025. Coding is up on LiveCodeBench, 66.5 to 65.5 for the 16-bit checkpoint, and it even edges the full-size 120B teacher on that one. QAH does not compensate for what quantization stole. It transfers information the earlier recovery stage never had time to move, because the student distills directly from the original, uncompressed model instead of from its own damaged copy.

The method also fixes the stability problem that plagues quantization-aware training. Under matched conditions on a smaller GPT-OSS 9B, QAH and QAT reach nearly the same peak score, 54.9 against 54.6. But QAH gets there in about 100 steps, roughly 7 times faster, and then holds. QAT peaks only near step 700 and then collapses, shedding almost 19 points by step 1,200. A QAT checkpoint needs careful early stopping or you ship a model that is already degrading. A sufficiently trained QAH checkpoint does not drift.

Tie that to what is happening on the serving side and the trend lines up. vLLM v0.28.0 shipped optional shared-expert sharding that saves up to 17 GiB of memory per GPU, letting larger mixture-of-experts models run on standard hardware. That is the same thesis from the other direction: the ecosystem is engineering around memory as the binding constraint, and QAH is the model-side complement. When half the parameters, a quarter of the weight memory, and a freed 17 GiB per GPU all point the same way, memory is the constraint, not capability.

THE COMPRESS-THEN-HEAL DECISION RULE

Call this what it is, a decision rule you can quote back and stick in a doc. The Compress-Then-Heal Decision Rule:

1. One: if you quantize only, standard healing applies. Run it, it works.
2. Two: if you compress and quantize, standard healing is the wrong tool. Use a recovery recipe that models both damage sources, structural compression and weight quantization.
3. Three: never assume the bigger checkpoint wins. Benchmark your healed model against the original, not against your expectations. The 7-of-9 result only shows up when you actually measure it.
4. Four: treat healing as part of the architecture, not a cleanup pass. The pipeline is compress, quantize, heal. Skipping the third step is how teams ship a worse model and then blame the quantization.

WHAT THIS MEANS FOR YOUR SERVING BILL

This is the next move in the same game PhantomByte has been tracking all along. Note 163, Your Late MoE Layers Are Disposable, showed you can mask half of late mixture-of-experts experts and keep 83.8 percent of quality. Note 141, Your Router Is the Moat, Not Your Model, showed Microsoft's 89 percent cost cut came from smaller models, quantization, and workload-specific training. Note 162, Your Frontier Model Is Wasting Tokens on Plumbing, made the same point from the token side. The savings live in the model you run, not the model you buy.

QAH is that running thesis sharpened. The model you serve is a build artifact you engineer, and the capability cliff between it and the frontier checkpoint is much smaller than the memory footprint difference. A 4-bit 60B model that beats its own 16-bit source on 7 of 9 benchmarks, at half the parameter count and a quarter of the weight memory, is a direct lever on cost per token. For operators, that is the whole game.

WHAT TO DO TODAY

If you ship a compressed and quantized model and heal it with a quantization-only recipe, stop and re-evaluate. That combination is exactly where this paper says standard healing fails.

Benchmark your current 4-bit model against its full-precision original. If you have never run that head-to-head, you do not know which one you are actually serving in effect.

Track Quantization-Aware Healing as it moves from paper to available recipes. The GPT-OSS 120B to 60B MXFP4 result is the proof of concept, and the weights are already out there as Hypernova-60B.

Recalculate your memory budget. Half the parameters at 4-bit plus a 17 GiB shared-expert saving is the difference between the hardware you are renting and the hardware you already own.

Put the Compress-Then-Heal Decision Rule in your model card. State what you damaged, how you healed it, and what you measured against. That last line, heal what you actually damaged then measure against the original, is what separates a deployable artifact from a bet.

THE UNCOMFORTABLE QUESTION

You are probably paying full-precision prices for a model you could rebuild at a quarter of the memory with better measured scores. How would you even know? Have you ever benchmarked the model you serve against the one you downloaded. Be honest. If the answer is no, the 7-of-9 result is not magic, it is just the first thing you measure when you finally look.