Compute at Scale: Growth, Limits, and AI Demand

Elsewhere, I’ve talked about “how small could you compress intelligence.” I never really gave a hard example of how small it can be, but one anchor is evolution: it’s been incredibly good at compressing intelligent systems into structured neural networks whose “design” is encoded in DNA. And DNA strands can contain anywhere from a few megabytes of data to several gigabytes, which gives you a rough sense of how much you can compress systems that are highly capable of interacting with the world and doing useful things.

But the other side of the question is: how big can we scale intelligence?

Right now, we’re in the middle of building massive new data centers—arguably the most ambitious infrastructure project in history—on a scale that dwarfs even things like building the International Space Station and particle colliders. It’s hard for most people to even comprehend. And the reason we’re doing it is that we’re seeing these incredible returns: the more intelligence we make, the more efficiencies we create, the more answers we have, and the more we’re able to do in this world.

So where does it go? What are the next steps?

I don’t think the most useful framing is “how much compute can you possibly build in the universe,” or “how much energy can you take from our sun and use to power compute.” I think the real question is: how much compute does humanity need to live out its ambitions?

Those ambitions will keep growing, so we’ll always want more compute. But I suspect it’ll look a lot like building a highway system: there’s a phase where a certain amount of highways is obviously useful to connect the country, and then after that you add more as needed. You do a huge build-out, you reach a point where you have more infrastructure than you need for your current set of ambitions, and then as you become more ambitious you find smarter ways to use what you already have—and you build out more strategically.

I don’t think we’re anywhere near the amount of compute I personally think I’d want for all the projects and ideas I have. But I also think the “right” amount is probably somewhere between where we are now and building a Dyson sphere around the sun to capture all available energy for compute.

If you forced me to put a number on it, I think something like 1,000× more compute than we have now would be pretty awesome—it would feel magical. Beyond that, it’s genuinely hard for me to even conceive what we’d do with it… although, at that point, we’d probably have new ideas.

So if you ask me what the upper limit is, I think it has more to do with our ambition—what we want to do—than with physics in the abstract. We already know there are limits in other domains. There’s an upper bound to attention: I can’t watch more than one TV show at once, and companies like Netflix and YouTube are constantly running into that ceiling as they compete for attention. I think our upper bound for compute needs is much, much higher than our attention bound, but probably not infinite—or at least not something that needs to grow exponentially forever after a certain point.