Tokens per Watt Is Your New Data Center Budget. 200 Demand Signals Cut a Live AI Factory From 4 MW to 3

On a sweltering August evening in Silicon Valley, the sun dropped and the air conditioning load spiked. Silicon Valley Power sent a signal to a running AI factory to pull back its power draw. Emerald AI's Conductor platform rescheduled the work that could wait and slowed what could be slowed. Facility draw fell from four megawatts to three. Nobody touched anything.

That was Emerald AI's first deployment across thousands of NVIDIA GPUs. Silicon Valley Power has since sent more than 200 demand signals to that factory. It worked every single time.

That is not a green energy story. It is a scheduling story. The AI factory that can turn itself down on command gets plugged into the grid years before the one that cannot.

Here is the thesis, blunt: your AI buildout is no longer gated by chips. It is gated by electrons. Flexibility is the currency you now buy interconnection with. First use definition, because the industry abuses the term: demand response means a grid operator pays you, or fines you, to change your power draw on short notice.

This note sits one layer above two already on this site. "Your Data Center Is a Control Problem, Not a Generation Problem" covered load-swing coordination after a fault. "Your GPU Is Useless Without a Wire" covered 800 VDC delivery inside the fence. Same physical layer, new commercial mechanism: the price of being controllable.

THE BINDING CONSTRAINT MOVED

The gas numbers tell you how fast the ground shifted under everyone's planning assumptions.

BloombergNEF now projects US data centers could consume about 18 billion cubic feet of natural gas per day by 2035. That is nearly double the same firm's forecast from nine months earlier. It is more than Germany and Japan combined. Onsite-powered campuses from Meta, Microsoft, Google, and Amazon account for 2.9 to 3.4 billion cubic feet per day of that, roughly as much as all data centers consume today including the gas burned for grid power.

The bigger number sits behind the meter. Grid-connected data centers are projected to drive an additional 15 billion cubic feet per day of power-sector demand, five times more demand growth through 2035 than all other grid-connected sectors combined. Burning one cubic foot of gas releases the equivalent of 60 grams of CO2 across extraction, processing, and distribution, so that added demand works out to roughly 1 million additional metric tons of greenhouse gas per day, about 12 percent of current total US emissions.

Now put the political path next to the physical one.

An early September poll from The New York Times and Siena University surveyed 1,503 likely voters. Asked whether they support or oppose construction of data centers to power AI, 61 percent said they were opposed and only 14 percent said they strongly support. Among opponents, 56 percent favored limits and 38 percent wanted a total ban outright. The stated reasons were environment and water usage at 32 percent, local community impact at 21 percent, and general distrust of AI at 18 percent.

Poway, California did more than answer a poll. Its city council voted unanimously to ban AI data centers, with Mayor Steve Vaus saying he does not care how much money is on the table. The ordinance takes effect October 6. In Philadelphia, city officials have identified two potential sites, and Philly Thrive's managing co-director Shawmar Pitts, who grew up beside what was then the East Coast's largest oil refinery before it shut down after a 2019 explosion, is leading a moratorium campaign.

Read those three items together and the shape is clear. Demand is rising on a curve nobody can move. The permitting path is narrowing. Siting is a municipal decision and there is no federal preemption standing in front of it.

Compute is purchasable. Interconnection is queued. That is the whole problem.

THE FLEXIBILITY-FOR-SPEED TRADE

On September 16, Google, NVIDIA, and Emerald AI launched the AI Energy Management Alliance. Eighteen launch partners joined them, including Anthropic, National Grid, AES, RWE, Constellation, and NRG. The pitch is a direct trade: data centers cut their power draw when the grid is strained, and in exchange grid operators connect those data centers faster.

Call it the flexibility-for-speed trade. It is the named mechanism of this article, and you should read it as a procurement lever, not an ESG press release.

The reason is arithmetic. AEMA points to a five to ten year backlog for connecting new AI facilities and claims flexible AI could free an extra 100 gigawatts from the existing grid. Emerald AI chief executive Varun Sivaram, in a Fortune commentary, wrote that the US grid runs at only about 50 percent of its capacity on average on his read, and cited a Brattle Group estimate that each 10 percent gain in utilization lowers electricity rates by about 3.4 percent. Google has committed 1 gigawatt of demand it can reduce through utility agreements across the country, per its head of advanced energy market innovation, Tyler Norris.

This is also the first time the largest AI hardware vendor and a hyperscaler have co-signed a load-flexibility pact with a grid signal provider, which is the part that matters structurally. That is my read, not a claim from the announcement.

What has to hold is verification. Sivaram said on a press call that facilities should qualify for faster or larger connections only if their ability to cut demand is "verifiable and enforceable." Hold them to that. A commitment you cannot measure at the meter is a press release wearing a hard hat.

Verification has a wire in it, and that wire is where the control-plane argument actually lands. Grid signals arrive over a standard protocol, and the one utilities most often speak is OpenADR, the open demand response standard maintained by the OpenADR Alliance. In its 3.0 release the whole interface is defined as an OpenAPI specification, with the utility or aggregator playing the Virtual Top Node and your facility playing the Virtual End Node. The signal comes down as an event object. The proof that you complied travels back up the same interface as a report. That report is your telemetry, and it is what the utility grades you on, which means the thing your facility is paid or fined against is not a human with a clipboard. It is an API your scheduler has to publish to. So the interface between the utility and your cluster scheduler is not a slide in a deck, it is an endpoint, an event ID, and a signed response your control plane emits under time pressure. If your stack cannot return a verifiable power-reduction report against the exact event it was handed, you are not flexible. You are promising.

The regulator track is already moving. In June, the Federal Energy Regulatory Commission directed regional grid operators to examine options for connecting large flexible power users. Texas, which is sitting on 474 gigawatts of data center interconnection requests, is finalizing rules to let controllable facilities connect sooner.

THE THREE-BUCKET SPLIT: SHED, SHIFT, SHIELD

This is the part you implement. Every workload in your AI factory lands in exactly one of three buckets, and the split has to be declared at the scheduler, not improvised at the breaker.

SHED is work you can drop or degrade when a signal arrives. Batch evaluations, synthetic data generation, non-urgent training runs.

SHIFT is work you can move in time. Checkpoint-restartable jobs, cross-region inference overflow, anything with a deadline measured in hours instead of milliseconds.

SHIFT is not free, and this is the cost most teams miss. Moving work in time means writing state and reading it back, and the state is not small. NVIDIA's own checkpoint math puts a 70B model at 782 gigabytes per full checkpoint, and most of that is not weights, it is the FP32 optimizer momentum and variance sitting in the training state. At 5 GB/s of storage throughput that single write leaves the GPUs idle for about 156 seconds. Tighten the checkpoint cadence far enough to answer a utility event on notice and the checkpoint tax starts eating the flexibility you just sold. So design the restart path deliberately: compressed or asynchronous checkpointing, a known write bandwidth per node, and a measured rehydrate time on one real job. SHIFT capacity you cannot restart inside the event window is SHED with extra steps.

SHIELD is work that never yields. Live serving, anything with an inference SLA attached.

SHIELD does not mean unchanged, it means protected, and an agentic workflow can still degrade on purpose. The pattern is a declared fallback ladder: drop the optional tool calls and keep the required ones, cap retrieval fan-out, shorten speculative decoding depth, cut the number of sampled candidates, and step the lower-stakes routes down to a smaller quantized model while the top-tier model holds the critical path. That is my design guidance, not a vendor claim, so treat it as engineering opinion and test it. Declare the ladder in the same place you declare the bucket, and measure what each rung costs in answer quality, because a degraded response that fails silently is worse for you than a slow one. This is the bridge between an inference SLA and tokens per watt: you are not choosing between serving and shedding, you are choosing how much quality to trade for how many megawatts.

The Conductor event is the pattern working end to end. The factory shed and shifted until draw went from four megawatts to three, and the shielded services kept running through all of it. NVIDIA product people describe exactly that hierarchy, lowest-priority jobs yielding while high-priority inference holds.

If you cannot name a bucket for a job, it defaults to SHIELD. That is the expensive answer, because SHIELD capacity is the capacity you just told the utility you cannot move.

THE NEW UNIT OF ACCOUNT: VALIDATED TOKENS PER MEGAWATT

At the AI Infra Summit in Santa Clara, NVIDIA's Ian Buck addressed more than 8,000 attendees, up from 3,500 the year before, and changed the scoreboard. NVIDIA's stated position is that the governing metric for AI infrastructure is shifting from peak performance to validated agentic tokens per megawatt.

Two numbers came with it. NVIDIA's DSX MaxLPS is claimed to deliver up to 1.4 times more tokens per megawatt through factory-wide power optimization. Lambda reported a 23 percent improvement in performance per watt using DSX MaxLPS, with Lambda's president of cloud services Dave Ward framing it as reclaiming stranded capacity inside a fixed power budget.

Caveat, honestly: those are vendor-run numbers. The shape of the claim is the signal, not the decimals. The shape says that power per unit of output is now the number infrastructure buyers will compete on, and that the vendors intend to define how it is measured.

So define it yourself before someone defines it for you.

WHAT TO DO TODAY

1. Pull your last month of jobs and sort every recurring workload into SHED, SHIFT, or SHIELD. If you cannot name a bucket for a job, it defaults to SHIELD.
2. Ask your utility or co-location provider one question: do you have a demand response program, and what does participation buy in interconnection priority.
3. Add a power-draw line to your deployment dashboard next to GPU utilization. You cannot schedule around a number you do not log.
4. Compute your own tokens per megawatt-hour on one production workload this week. A rough number you measured beats a nameplate spec you are quoting.
5. Inventory which batch workloads are checkpoint-restartable, and time an actual rehydrate. SHIFT capacity you have not built restart for is SHED you have not admitted to.

THE UNCOMFORTABLE QUESTION

You spent the last two years optimizing your model architecture to save tokens. Are you spending even one hour this quarter optimizing when your factory draws them? The grid does not care how clever your scheduler is on paper. It cares whether you can be turned down.