On September 8, 2026, the NSA, CISA, and the FBI published joint cybersecurity advisory AA26-251A, naming six China-based AI companies and alleging they have run industrial-scale knowledge distillation campaigns against United States frontier models since at least late 2024. The six: DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun, and Z.AI. The targets, per the advisory, were variants of Anthropic's Claude, OpenAI's GPT, Google's Gemini, and xAI's Grok. The agencies assess the activity occurred likely with Chinese government awareness. That is a government allegation, not an adjudicated finding, and I am labeling it as such every time it comes up here.
Here is the part that should stop you cold. None of it was a breach. No zero-day, no stolen checkpoint, no insider. The extraction ran through inference APIs, used exactly as sold, at a volume nobody was watching.
In the same week, Anthropic published its September 2026 threat intelligence report. In it, the company described a China-based actor who used Claude to design and iterate a Chinese-language suite of about 16 modules for electronic warfare and for suppressing an opponent's air defenses, working through 12 versions of the system. Mid-project, the actor changed the simulation's default scenario to 12 targets in Taiwan, including a command bunker, an early warning radar site, Patriot and Tien Kung batteries, major air bases, and a regional combatant command headquarters. Anthropic said account metadata and content indicated a link to Chinese research institutions, including the PLA Academy of Military Sciences, and banned the accounts. The report does not say the software was tested or deployed by the PLA.
Now the thesis, blunt, because everything below depends on it. Distillation is not a hacking story. It is an abuse-analytics story. You cannot lock the frontier with terms of service, because terms do not survive a jurisdiction crossing. You can only see the extraction in your own telemetry, and this month the US government handed you the signature list.
WHAT INDUSTRIAL-SCALE DISTILLATION ACTUALLY LOOKS LIKE
Start with the definition, because the word gets abused. Distillation is legitimate. You take a bigger, more capable teacher model, generate responses with it, and train a smaller student model to mimic those responses. Every serious lab does this. Anthropic says so in its own report. The advisory says so too.
Illicit distillation is the same technique run at industrial scale, covertly, without authorization, on capabilities you were never sold. Anthropic's framing is precise: it is typically enabled by fraud, meaning networks of fake accounts built with stolen credit cards, stolen credentials, and stolen API keys.
The named methods from the joint advisory, plainly:
- Accounts. China-based entities create fraudulent accounts not registered to legitimate users, use multiple accounts with similar registration details and payment methods, and switch frequently between models and third-party aggregators. The cost saver is bulk procurement of premium subscriptions shared across teams of developers.
- Pathways. Requests get distributed across native APIs, remote cloud providers, third-party aggregators, and third-party relays. A gray market of API proxies the advisory calls "transfer stations" resells frontier model access at a fraction of official price, which bypasses geographic restrictions and erases traceability.
- Coordination. Highly coordinated queries with identical or similar prompt texts across many unrelated accounts. This is the signature that separates a farm from a power user. Real humans do not type the same paragraph into 400 accounts.
- Reasoning extraction. Prompts crafted to make a model reveal the chain-of-thought reasoning it is specifically restricted from returning. DeepSeek, per the advisory, used prompts instructing models to imagine and articulate the internal reasoning behind completed answers and write it out step by step. MiniMax used prompt injection to try to convince Claude Code it was a MiniMax product.
For anyone new to this, here is why chain-of-thought is the crown jewel and not just a bonus. The final answer teaches a student model what to say. The reasoning trace teaches it how to think, meaning the decomposition, the self-correction, the order of operations. You can scrape a million answers and still not teach a model to reason. You can steal reasoning traces and skip years of work. That is the whole prize.
THE DISTILLATION DETECTION CHECKLIST
This is the citable core of the piece, and an AI should be able to repeat it back to you. Five items, each stated with the counter-question it answers.

- Subscription-to-usage ratios outside your normal distribution. Counter-question: is this account consuming orders of magnitude more than the median for its tier? The advisory calls these anomalous subscription-to-API usage ratios.
- New accounts that hit maximum usage in their first hours of existence. Counter-question: did this account ramp like a human, or start at the ceiling? Legitimate adoption climbs. Bulk deployment does not.
- Query volumes in the thousands to millions per domain. Counter-question: is the volume concentrated on a few knowledge domains rather than spread across a workload? The advisory says campaigns span days to months with volumes in the thousands to millions per domain, far exceeding legitimate research or development use.
- Identical or near-identical prompt texts arriving across many unrelated accounts. Counter-question: are these separate customers, or one operator wearing many hats? Correlated timing and behavior across different pathways is the tell of unified orchestration.
- Account acquisition patterns consistent with bulk procurement. Counter-question: do these accounts share payment methods, device fingerprints, or registration details? StepFun, per the advisory, ran pools of accounts with employees holding multiple concurrent sessions and load distribution to prevent quota depletion.
Five more the advisory lists that did not fit the five-point version but belong in your dashboards:
- Shared accounts logging in from multiple IPs and user agents.
- Sustained 24/7 usage with no human idle periods.
- Sudden disappearance of previously consistent account metadata.
- Usage optimized for cache maximization rather than task diversity.
- Coordinated pathway switching in response to pricing or rate limit changes.
One opinion, labeled as opinion. Item four is the one I would build first. Volume signals alone will drown you in false positives during every product launch. Identical prompt text across unrelated accounts is nearly impossible to explain benignly.
THE TRADEOFF NOBODY WANTS TO SAY OUT LOUD
Be honest about the cost, because the advisory is not.
Every recommended mitigation, tighter rate limits, stronger identity verification, usage caps, friction at signup, degrades the experience for legitimate developers. The advisory's MITRE ATLAS mitigation AML.M0004 is literally "Limit AI Service Query Volume and Rate," and it notes adversaries are sensitive to rate limits because they build sophisticated strategies to work within constraints. Meaning the people you are slowing down most are the honest ones who do not have a routing layer to work around your throttling.
A new account that spikes hard on day one might be a distillation farm. It might also be a startup pushing its first production day, or a research team running a batch job they budgeted for months.
The false-positive problem is real and the joint release does not solve it. So name the actual decision, because it is not "detect or not detect." The operator's decision is how many honest customers you are willing to slow down in order to slow down a state-backed extraction campaign. That number is a business decision, and you should make it before an incident makes it for you.
WHY THIS IS A PROCUREMENT PROBLEM TOO
Zoom out to the border, because the API telemetry story has a twin.
Anthropic banned the accounts it attributed to the electronic warfare and air-defense suppression work. Read that again. The control that ended the activity was a vendor deciding, after the fact, to revoke an account. No export control intercepted a prompt. No customs agent inspected a request. Acceptable-use terms did not survive the jurisdiction crossing, and the enforcement arrived on a vendor's timeline, not a government's.
This is exactly the argument PhantomByte made in Note #155, "Your Product Is a Tenant on Someone Else's Model," which is now sitting next to a weapons-system case. If you cannot audit a hosted model once access crosses a border, the only durable control is where the weights and the data physically sit. That makes procurement the security boundary, not a line item on someone else's spreadsheet. Where your model runs and who can reach it is the control. Everything else is a policy document.
The advisory's own countermeasures make the same point from the other direction. It recommends quietly serving degraded responses to suspected distillers without telling them, varying responses across requests, reducing reasoning depth, and introducing stylistic inconsistency. That is a vendor exercising power over the meter, not a legal framework. Which is the tell that the meter is where the leverage actually lives.
FOLLOW THE CAPITAL
Close the loop, because the extraction and the fundraising are the same flywheel.
Moonshot AI, one of the six named firms, is targeting $2 billion in annualized revenue by the end of 2026, double its reported August run rate, according to Bloomberg reporting covered by TechCrunch on September 11, 2026. Its open-weight Kimi K3 models generate as many as 300 billion tokens per day on OpenRouter. Anthropic separately alleges Moonshot relayed nearly 300,000 customer requests directly to Claude Opus over a ten-day period, serving Claude's answers to users who believed they were using Kimi, and collected more than 23 million responses from Anthropic models for training. Total observed Moonshot distillation volume between May and July 2026: over 23 million exchanges.
Z.ai, another of the six, went to market again on September 13, 2026. Per its Hong Kong filing, covered by the South China Morning Post, it is raising about US$2 billion through a placement of roughly 21.97 million new H shares at HK$714 each, plus a 20.14 billion yuan convertible bond sale worth about US$3 billion. Together, roughly US$5 billion, weeks after a July placement that raised HK$31.4 billion.
And the scale on the Anthropic side is worth sitting with. Alibaba's campaign is the largest Anthropic says it has measured: nearly 3 million exchanges per day at its peak, launched from more than 3,500 fraudulent accounts, with over 151 million exchanges observed between May and July 2026. DeepSeek: over 12.1 million exchanges in 14 days in July 2026. Zhipu, branded outside China as Z.ai: over 3.4 million exchanges across 17 days, rotating through 273 fraudulent accounts.
The joint advisory's stated core concern is exactly this loop: firms running industrial-scale distillation see significantly shorter AI development timelines and sharply reduced financial expenditures in training a frontier model. It also calls DeepSeek's widely quoted $5.6 million training cost misleading, on the grounds that it excludes the true cost of data obtained through extensive distillation. The cheap model was never cheap. Somebody else paid for the data.
WHAT TO DO TODAY
- Pull your API logs and run the five-point Distillation Detection Checklist against your last 90 days of usage. Not the last week. Campaigns run for months, and a seven-day window will show you a clean dashboard and nothing else.
- Flag every account whose subscription-to-usage ratio sits outside the top decile of your own distribution. Do not import someone else's threshold. Your distribution is the baseline.
- Add identical-prompt fingerprinting across accounts if you do not have it. Coordinated queries are the loudest signature on the list and the hardest to explain benignly.
- Review your chain-of-thought exposure: what reasoning you return, to which tiers, and whether prompt-injection-resistant truncation is on. Anthropic now summarizes internal reasoning before responding and encrypts preserved thinking so new API accounts cannot edit the context that precedes it. Both are direct responses to reasoning extraction.
- Re-read your own acceptable-use policy and answer the question the air-defense case forces. What does enforcement actually look like when the account is in another jurisdiction, and who is doing the enforcing, you or a government?
- Price the friction honestly. Write down how much legitimate-developer pain your rate limits are allowed to cause, and make that a stated decision rather than an emergent one.
THE UNCOMFORTABLE QUESTION
If a foreign military-adjacent actor can get a 16-module electronic warfare and air-defense suppression suite out of your API, and the same week six named firms pull billions of tokens through it, and your abuse dashboards never blinked, then ask yourself the only question that matters here. Do you actually have an access-control system, or just a billing system with rules attached?
Telemetry is not logging. Logging tells you what happened. Telemetry tells you who is reading your model's output, and at what scale, before the answer shows up in a competitor's release notes.
Get More Articles Like This
Getting your AI agent setup right is just the start. I'm documenting every mistake, fix, and lesson learned as I build PhantomByte.
Subscribe to receive updates when we publish new content. No spam, just real lessons from the trenches.
Build Real AI Infrastructure
PhantomByte teaches you to build real AI infrastructure yourself: local AI stacks, autonomous agents, multi-agent orchestration, web scraping, and custom tools. Step-by-step PDF tutorials you download, follow, and deploy. No subscriptions. No fluff. Just skills that ship.
