Most of us have become comfortable with today’s AI pricing: a few dollars per month for a chatbot, a few cents to generate an image, and sometimes even free. It feels almost magical. But is AI actually cheap? The answer depends on who is paying the electricity bill.
Let’s Start with the AI Chip
Modern AI models are trained and served using specialized GPUs designed for massive parallel computation. As each generation becomes more powerful, it also consumes dramatically more power. Below is a table for comparison:
| GPU | Typical Power Consumption |
|---|---|
| NVIDIA A100 | ~400 W |
| NVIDIA H100 | ~700 W |
| NVIDIA GB200 | ~1,200 W |
| NVIDIA B200 | ~1,400 W |
As you may know, a modern home air conditioner typically consumes around 250–300 W while operating. Here, we can see that a single cutting-edge AI GPU can consume roughly five times what one air conditioner consumes. And that’s just one chip.
The Rack of AI Chips
An AI chip rack is an ultra-dense, multi-node server cabinet built to pack hundreds of AI accelerators (like GPUs or TPUs) into a single interconnected system. A single rack can house approximately 72 high-end GPUs, 36 server-grade CPUs, terabytes of high-bandwidth memory, petabytes of NVMe storage, and ultra-fast networking hardware capable of handling hundreds of gigabits per second. These components work together as a single distributed computer, continuously exchanging enormous volumes of data while training or serving AI models.
Now imagine two of these racks operating side by side, 24 hours a day. Together, they can draw well over 100 kilowatts of power – enough electricity to supply more than a dozen households simultaneously. And that’s just the computing hardware.
The Heat
According to some research, GPUs are inefficient at turning electricity into computation: roughly 60% of their energy becomes heat. That heat must be removed by a cooling system, just like every other engine. Cooling systems often require around half as much electricity as the computing equipment itself, meaning that every watt used for AI computation demands somewhere around 0.5 additional watts simply to keep the hardware from overheating. In large AI data centers, cooling is no longer a supporting system; it is one of the largest energy consumers.
Scale It Up
Now scale this up: take xAI’s Colossus supercomputer as an example. This data center houses approximately:
- 500,000 AI GPUs
- Thousands of CPUs
- Massive storage system
- Ultra-fast networking
The computing hardware alone is estimated to require roughly 830 MW. After adding cooling and infrastructure overhead, the total demand approaches 1 gigawatt (GW). That is a massive amount of electricity.
How Big Is 1 GW?
One gigawatt is difficult to visualize; here are a few comparisons:
- 1 GW = half of the maximum generating capacity of Vietnam’s Hòa Bình Hydropower Plant.
- 1 GW = enough electricity to power hundreds of thousands of homes simultaneously.
- 1 GW = electricity demand of a small country
To simplify, one AI data center can require the entire output of a power plant. This is why we have heard that nuclear power plant projects are being restarted.
Which Parts Consume Electricity the Most?
The largest energy cost occurs during the training phase. According to some research, electricity consumption for training is massive:
| Model | Estimated Electricity |
|---|---|
| GPT-3 | ~1.3 million kWh |
| GPT-4 | ~50 million kWh |
| Grok | ~310 million kWh |
Training an AI model like what we see today can consume enough electricity to supply around 4,000 households for an entire year. After months of computation, the finished model can often be stored on a device that is no larger than a USB drive. Thousands of GPUs, months of computation, and hundreds of millions of kilowatt-hours are all compressed into a few gigabytes!
Even after training is complete, every interaction requires computation.
A typical text question consumes less than 1 Wh—tiny compared to training—but the numbers become enormous at a global scale. Let’s say ChatGPT has around 100 million daily active users, and each user asks 10 questions per day; that’s roughly 1 billion prompts every day. Let’s say each prompt consumes 0.5 Wh; generating answers alone would consume about 500 MWh of electricity daily—enough to power over 10,000 homes for a day. And for more complex tasks such as image generation, long reasoning sessions, or video creation, substantially more energy per request is required:
- Creating an image can require roughly four times the computation of a text response.
- Generating video may require 25 times or more.
As AI becomes increasingly multimodal, the demand for electricity rises accordingly.
AI Is Not Really Cheap
From the user’s perspective, AI feels incredibly cheap. But behind every prompt, an AI system uses:
- Hundreds of thousands of GPUs.
- Gigawatts of electrical power
- Massive cooling system
- Entire power plants are dedicated to computation
The low price we pay is possible for now only because huge companies such as Google, Amazon, Meta, and X are investing billions of dollars in infrastructure and do not require instant revenue in return. These companies are competing to keep users on their platforms, and we—prompters—are the beneficiaries of that.
But next time, when asking an AI for answers to generate something, it’s worth remembering that the response may appear instantly on your screen—yet somewhere, an industrial-scale data center is consuming enough electricity to power an entire city.
Build – Secure – Evolve with the-tech-lead.com