Most of us have become comfortable with today’s AI pricing. A few dollars per month for a chatbot. A few cents to generate an image. Sometimes even free. It feels almost magical. But is AI actually cheap? The answer depends on who is paying the electricity bill.
Lets start with the AI Chip
Modern AI models is trained & serving using specialized GPUs designed for massive parallel computation. As each generation becomes more powerful, it also consumes dramatically more power. Below is a table for comparison:
| GPU | Typical Power Consumption |
|---|---|
| NVIDIA A100 | ~400 W |
| NVIDIA H100 | ~700 W |
| NVIDIA GB200 | ~1,200 W |
| NVIDIA B200 | ~1,400 W |
As you can know, a modern home air conditioner typically consumes around 250–300 W while operating. Here we can see a single cutting-edge AI GPU can consume roughly five times what one air conditioner consumes. And that’s just only one chip.
The Rack of AI Chips
An AI chip rack is an ultra-dense, multi-node server cabinet built to pack hundreds of AI accelerators (like GPUs or TPUs) into a single interconnected system. A single rack can house approximately 72 high-end GPUs, 36 server-grade CPUs, terabytes of high-bandwidth memory, petabytes of NVMe storage, and ultra-fast networking hardware capable of hundreds of gigabits per second. These components work together as a single distributed computer, continuously exchanging enormous volumes of data while training or serving AI models.
Now imagine two of these racks operating side by side, 24 hours a day. Together, they can draw well over 100 kilowatts of power – enough electricity to supply more than a dozen households simultaneously. And that’s just only the computing hardware.
The Heat
According to some researches, GPUs are inefficient at turning electricity into computation: roughly 60% of their energy becomes heat. That heat must be removed by a Cooling system, just like every other engines. Cooling systems often require around half as much electricity as the computing equipment itself, meaning that every watt used for AI computation demands somewhere 0.5 watt additional power ,simply to keep the hardware from overheating. In large AI data centers, cooling is no longer a supporting system, it is one of the largest energy consumers.
Scale it up
Now scale this up. Take xAI’s Colossus supercomputer as an example. This data center houses approximately:
- 500,000 AI GPUs
- Thousands of CPUs
- Massive storage systems
- Ultra-fast networking
The computing hardware alone is estimated to require roughly 830 MW. After adding cooling and infrastructure overhead, total demand approaches 1 gigawatt (GW). That is a massive amount of electricity.
How Big Is 1GW?
One gigawatt is difficult to visualize. Here are a few comparisons:
- 1GW = a half of the maximum generating capacity of Vietnam’s Hòa Bình Hydropower Plant.
- 1GW = enough electricity to power hundreds of thousands of homes simultaneously.
- 1GW = electricity demand of a small country
To simplify, one AI data center can require entire output of a power plant. This is why we heard that nuclear power plant projects are restarted.
Which parts consume electricity the most ?
The largest energy cost occurs at training phase. According to some researches, electricity consumption for training is massive:
| Model | Estimated Electricity |
|---|---|
| GPT-3 | ~1.3 million kWh |
| GPT-4 | ~50 million kWh |
| Grok | ~310 million kWh |
Training an AI model like what we see today can consume enough electricity to supply around 4,000 households for an entire year. And after months of computation, the finished model can often be stored on a device that is no larger than a USB drive. Thousands of GPUs, months of computation, hundreds of millions of kilowatt-hours, all compressed into a few gigabytes !
Even after training is complete, every interaction requires computation.
A typical text question consumes less than 1Wh- tiny compared to training – but the numbers become enormous at global scale. Let say ChatGPT has around 100 million daily active users, and each user asks 10 questions per day, that’s roughly 1 billion prompts every day. Let say each prompt consumes 0.5 Wh, generating answers alone would consume about 500 MWh of electricity daily—enough to power 10.000+ homes for a day. And for more complex tasks such as image generation, long reasoning sessions, or video creation, require substantially more energy per request:
- Creating an image can require roughly 4× the computation of a text response.
- Generating video may require 25× or more.
As AI becomes increasingly multimodal, electricity demand rises accordingly.
AI Is Not Really Cheap
From the user’s perspective, AI feels incredible cheap. But behind every prompt, an AI system uses:
- Hundreds of thousands of GPUs
- Gigawatts of electrical power
- Massive cooling systems
- Entire power plants dedicated to computation
The low price we pay is possible for now only because huge companies such as Google, Amazon, Meta, X, etc .. are investing billions of dollars in infrastructure and does not require instant revenue back. These companies are competing to keep users on their platform and we – prompters – just are beneficial from that.
But, next time, when asking an AI answers to generating somethings, it’s worth remembering that: The response may appear instantly on your screen – but somewhere, an industrial-scale data center is consuming enough electricity to power an entire city.