Key Takeaways
Megawatts describe the ask: A capacity figure tells you how much power a facility will draw from the grid.
Tokens per megawatt aggregate the whole stack: The ratio folds in cooling overhead, accelerator generation, serving software, and utilization.
NVIDIA puts a fully optimized AI factory at 2.6 times the tokens-per-second-per-megawatt of an unoptimized one. Scheduling and runtime tuning carry a large share of that.
Facility overhead sets the ceiling before compute delivery starts: At the 2026 industry-average Power Usage Effectiveness (PUE), 100 MW of IT load requires 152 MW from the grid.
Capacity Announcements Skip AI Factory Output
NVIDIA published a technical argument in March 2026 that the AI buildout had reached a hard ceiling: power is the ultimate constraint, and every AI factory operates within it. The industry still announces capacity in megawatts.
A megawatt is an input. Tokens per watt is an output, and the gap between them is where most of the interesting engineering sits. The Uptime Institute 2026 global survey puts the industry average annual PUE at 1.52, essentially flat for seven years, while modal rack density passed 11 kW for the first time. Power is getting denser faster than facilities are getting efficient.
Identical grid draw tells a buyer nothing about which facility serves more requests. The difference shows up in cooling design, accelerator generation, and how well the scheduler packs work onto silicon.
Procurement teams that once asked how many GPUs a provider had now ask what those GPUs produce per unit of energy.
Tokens Per Megawatt Is Becoming the Key AI Factory Metric
The metric is output tokens divided by the energy it took to produce them. NVIDIA reports 50 times higher throughput per megawatt for Blackwell generation chips than Hopper on DeepSeek-R1, using SemiAnalysis InferenceX data, and 35 times lower token cost in the same comparison.
Vera Rubin generation GPUs are quoted at roughly ten times the inference throughput per megawatt of Blackwell on Kimi K2 at 32K context and 8K output, and NVIDIA separately describes 30 times more work per watt for agentic workloads on the Vera Rubin NVL72.
Prefill and decode behave differently: Reading a long prompt is compute-bound work. Generating the answer is memory-bandwidth bound. A single efficiency number averages two phases with very different power profiles.
Interactivity level changes everything: The same cluster produces far more tokens per watt at relaxed latency than at a tight per-user response target. Vendor figures state that setting, and buyer quotes frequently don’t.
Facility Overhead Caps AI Data Center Efficiency
Every data center spends part of its power on itself. Cooling, power conversion, and lighting all draw electricity before a single GPU starts working. The industry tracks this with Power Usage Effectiveness (PUE), which is the total power a building pulls from the grid divided by the power that reaches the servers.
A PUE of 1.0 would mean every watt goes to compute, and no real facility gets there. The 2026 industry average is 1.52, so a typical building spends about half a watt on overhead for every watt its servers use. NVIDIA designs its Blackwell halls for 1.25 and its Vera Rubin halls for 1.1.
How PUE Converts Grid Power Into Inference Throughput
The gap between those numbers is bigger than it looks. Take a fixed grid connection of 152 MW:
At the 1.52 industry average, it powers 100 MW of GPUs.
At a PUE of 1.1, the same connection powers about 138 MW of GPUs.
That’s roughly 38% more compute from the same power supply, without a single new grid connection. When power is the scarcest input in AI, a more efficient building is effectively extra capacity.
Most of that difference comes from cooling. Air cooling struggles as racks pack more GPUs into the same space, and direct liquid cooling makes lower PUE figures achievable. Cooling fixes the building, but it doesn’t make the GPUs inside do more useful work, which depends on software and scheduling.
Where you measure power also changes the headline number. A tokens-per-megawatt figure measured at the servers leaves out the building’s overhead, so it’ll always look better than one measured at the grid connection.
How Aethir ACCELERATE Is Measured on Tokens Per Megawatt
An idle GPU still draws power and costs money while producing zero tokens.
Aethir ACCELERATE plans to bring GPU compute online in facilities built for high-density AI training and inference. Those facilities will plug into the orchestration layer that already routes enterprise workloads across 430,000 GPU containers in 94 countries. New capacity meets existing demand, so the GPUs start busy instead of waiting for customers to find them.
A finance team cares about cost per million tokens, and every layer above shows up as a price. Two providers running the same GPUs can quote very different prices, usually because of utilization.
A tokens per megawatt or cost per million tokens figure is only comparable between providers when three details match:
The workload: Long prompts and large models put far heavier loads on hardware than short ones. A figure measured on a small model with short requests will look much better than real production traffic.
The response speed: A cluster produces more tokens per watt when users can wait than when every answer has to come back fast. Chatbots and AI agents need quick replies, so figures measured at relaxed speeds overstate what a live product will get.
The utilization: A number measured on a fully loaded cluster is best-case. The real cost depends on how busy the hardware stays across a normal week.
McKinsey reports grid connection waits of more than four years, so getting more out of already-running capacity is the faster, cheaper half of the problem. Tokens per megawatt has become a key metric for judging an AI data center’s realistic performance and pricing.
Frequently Asked Questions
What is tokens per megawatt?
Tokens per megawatt measures how many output tokens a facility produces for each megawatt of power it consumes. It combines cooling overhead, accelerator generation, serving software, and utilization, so it describes the whole system rather than the chip alone.
Do tokens per watt depend on the model being served?
Heavily. NVIDIA quotes its Blackwell and Vera Rubin comparisons against named models at stated context lengths, because a long-context request and a short one put very different loads on the same rack. No single number exists across all workloads.
What does Aethir ACCELERATE mean for AI factory output?
ACCELERATE brings GPU compute online in facilities purpose-built for high-density compute and built for AI training and inference workloads. Because those facilities sit behind an orchestration layer already carrying enterprise traffic, the utilization term that dominates tokens per megawatt starts higher than it would for capacity built speculatively.
