AI Looks Like Software. It's Actually Heavy Industry — What Building an AI Service Taught Me About the Cost of Intelligence

I'm building a service on top of AI. At first, all I cared about was what I could get it to do. How well does the model read what a user actually wants? How much of a messy, multi-step job can it take off their hands?

Then I ran the numbers on what happens at 100 users. Then 1,000. Then 10,000. The question changed. It stopped being "what can AI do" and became "how many people can I keep doing this for, month after month?"

Regular software costs money to run too. But AI cost behaves differently. The longer a conversation runs, the more context there is to chew through on every turn. Hard tasks don't finish in one response — they loop through reasoning, retrieval, and tool calls. When an agent fails halfway and retries, the user sees one request. I get billed several times.

Route everything to the best model and quality goes up. Sustainability goes down with every new signup. Route everything to a small model to save money and accuracy starts wobbling, and so does the experience. On top of that: how much do I give away free, where do the caps sit, which tasks earn the expensive model, and how do I stop runaway retries and abuse from eating the budget?

Designing an AI service turns out to be an exercise in budgeting intelligence.

Scale that from 10,000 people to 800 million

Take my little service's problem and blow it up to planetary scale. The question gets more interesting.

Right now, only a sliver of the world's population uses frontier AI seriously. What happens when 10% of humanity — roughly 800 million people — uses it as part of daily life?

Headcount isn't what matters. Frequency and depth are. A few short questions a day is one thing. A handful of personal agents grinding away around the clock is something else entirely.

Agentic systems are the sharp edge here. One human request turns into planning, searching, checking the result, and calling the model again and again. One click on the surface, dozens of inference passes underneath. Add routine image and video generation and the resource curve bends harder.

The IEA projects global data center electricity use will roughly double, from about 485 TWh in 2025 to about 950 TWh in 2030 — around 3% of world electricity demand by then. AI-focused data centers triple their consumption over the same window. Three percent sounds survivable as a share. The trouble is that demand clusters geographically. Having a power plant nearby doesn't help if you can't get a grid connection, transformers, cooling, and land in time. Without those, the facility doesn't run.

AI looks like pure software because it happens on a screen. It runs on silicon, high-bandwidth memory, generation capacity, transmission lines, chillers, and water. It's software and heavy industry at the same time, and it keeps drifting toward the second one.

Won't efficiency fix this?

Efficiency really is improving. Smaller models keep landing. Purpose-built silicon keeps getting faster. The cost of producing the same answer will keep falling. Caching common answers, sending easy work to cheap models, and reserving the expensive model for genuinely hard problems — all of that is getting better too.

But cheaper per unit doesn't mean less consumed overall. When AI gets cheap enough, it starts taking on work that never penciled out before. Cut the energy per inference by 10x, watch usage go up 100x, and total consumption rises anyway.

Coal use went up, not down, after steam engines got more efficient. Jevons applies here too. Cheap intelligence doesn't conserve intelligence — it manufactures new demand for it.

Companies won't stop at one AI handling customer support. They'll want agents running in parallel across sales, engineering, design, analytics, and accounting. Individuals will want more than calendar management and search — they'll want something watching their work and doing pieces of it. The price per call drops while total compute the world burns goes vertical.

The real edge is allocating intelligence

So is a future where everyone has unlimited access to today's top-tier model realistic? Not soon, from where I sit.

Tiered is the likelier shape. Small on-device models on phones and laptops handle routine work. Mid-tier cloud models take the ordinary jobs. The frontier model gets called only for problems that actually need it. Anything that isn't urgent waits for a window when power and servers are slack.

Operators will stop giving every request the same compute and start allocating different amounts of intelligence based on how valuable and how hard the task is. Users will pay by frontier inference count, agent runtime, or queue priority. Today we sell plans by storage and bandwidth. Tomorrow we may sell them by how much intelligence you're allowed to spend.

This creates a new kind of gap. A company with more compute tests more hypotheses in the same hour, ships more content, automates more of its operation. A country with reliable power and chip supply delivers cheaper intelligence across its whole economy than one without.

Competitive advantage in AI won't come from owning the best model. It'll come from deciding, inside a fixed resource envelope, which problems get how much intelligence.

I'm hitting the same wall on a much smaller scale. Features I can hand out freely to 100 people need a different architecture at 10,000. Simple tasks go to rules and plain old code. Repeated results get reused. The expensive model gets called only when nothing else will do. There's a line between what's technically possible and what's commercially survivable, and you have to draw it.

For me the question is how to cut cost. At global scale, the same question becomes power, chips, and grid capacity.

Data used to get called the oil of the 21st century. In the AI era, data alone doesn't produce intelligence. You need chips to compute on, power to run them, and infrastructure to deliver the result.

The scarce resource of the AI era isn't information. It's compute you can reach for the moment you need it — which is to say, electricity that can be turned into intelligence.

References

• IEA, Key Questions on Energy and AI