This piece is the first in a new series from the Institute for Progress (IFP), called Compute in America: Building the Next Generation of AI Infrastructure at Home. In this series, we examine the challenges of accelerating the American AI data center buildout. Future pieces will be published
Why not build data centers where there is lots of power and cooling is less of an issue due to cold outdoor temperatures - like the James Bay area of Quebec?
Some of the requirements on a high level data center in the charts above included proximity to a major metro area and to an airport - probably this is because you want emergency technicians and backup parts to be able to get to the location quickly if anything bad happens. They also mentioned proximity to major internet hubs, so that latency of communications would be low.
If there's enough of an advantage, it might be possible to pay some people to live full-time near the datacenter to be on call, but unless there's some new construction of major internet nodes in these areas, that isn't going to be much of a benefit.
I could imagine Iceland becoming a major location though, with low temperatures, cheap electricity, location on some major transatlantic cables, and an established town where you could pay some technicians to live full time.
I'm curious how much the outdoor temperature really matters. I mean you still need active cooling in the summer and piping outside air deep into the structure would take a fair bit of space and engineering not to mention creating another potential point of failure. Also you can also use a large body of water and pipe in cold water to a heat exchanger (or maybe both aren't worth it).
Of course that only works for the training, you want servers relatively near the user.
Outdoor temperatures matter less, but there is excess power in Quebec. There has been a political battles for years over building transmission lines from Quebec south to New England. So why not put the data centers in Quebec and run fiber south which no one really cares about, unlike high voltage transmission lines.
You may want servers close to the users but latency may not be that much of an issue so a few more ms may not be a big deal if you have users in Boston-NY hitting a server in James Bay rather than Virginia.
Agreed. I'd like to use full immersion cooling with some kind of outdoor heat exchange to run a Stirling generator. Summer is still a problem, as is telecom.
I'm kinda surprised at the NIMBY issue. I mean a data center is in some sense the perfect neighbor as they will occupy high value real estate and have virtually no burden in terms of transit nor burden most city services.
For example, Quebec does not like data centers and cryptomines because they use a lot of electricity, provide few jobs, and because of tax policies the companies are almost always headquartered in Toronto or DE.
That makes sense, but it seems less like a NIMBY issue than an electricity and tax concern. I guess it's just a matter of terminology but that's not what I think of when I hear NIMBY.
HV transmission through residential areas is a major pushback. N Virginia growing from 3GW of datacenter power to 10 GW by 2035 is driving more 500kV routes needing new rights of way.
Trying to power 24 x 7 datacenters with intermittent renewables and battery storage, which would be primarily PV in N Virginia requiring tens of thousands of acres of farmland (several counties worth).
If you think NIMBY's object to HV transmission, wait until there are proposals for nuclear reactors in a residential neighborhood.
In the case of northern Quebec, running power lines and batteries through residential areas would be less of a problem, because the region is sparsely populated. And, of course, cold and with abundant hydropower.
The more lucrative a project is, and the more money at stake, the *more* sense it is to become a NIMBY and demand a toll/tax/"my fair share" for allowing the project to even exist, when I could block it and threaten the entire project. The company is willing to spend $10 billion on this new data center project? Good, that means I can demand $1 billion and not get laughed out of the room. The data center project isn't actually *that* objectionable? Who *cares* about that, it was never about the project itself, but the money I can extort from it. The project could be an orphanage for kittens and it'd *still* make sense for me to shake it down for cash, every single last cent it can give, as long as the law supports my NIMBY-ism.
Before leaving negative thoughts, I have to say I'm a big fan and love your posts! I devour all your posts with great interest ^^
But I really was not a fan of this one. It reads like an opinion piece or ad copy, and takes a very strong pro-USA and pro-AI stance. I guess your other posts aren't completely neutral on all topics, but I feel like they're much more evenhanded.
Anyway, happy you're getting writing gigs; I just hope similar ones won't be overly frequent in this feed in the future. :)
We have not yet seen the real value of AI. So far it feels like a huge investment when we have much more pressing matters to attend to.
Climate change ? Education ? Health ?
AI as is has an enormous potential but will be used for foolish reasons and with no social integration at all. We only start to realize the huge impact social media and the digital world has on our social norms, our work and our culture…
AI is already taking jobs, by no means very interesting or exciting jobs, but still. How do we guarantee that AI won’t be just one more capitalist optimization of our society that leads rich to be richer and poor to be poorer and our environment to be a collateral damage AGAIN ? (Exaggerated but you get my point…)
We need a new social contract around AI usage, it’s crucial. As usual, not a skeptic regarding the technology but rather the use we make of it, seeing who’s leading that market I’m already shaking.
Something that would have been interesting to compare to was the crypto boom stupidity - another compute-intensive energy gobbling false prophet of the future.
This is a nice piece! I am hopeful the series will include a post on the nature of how data centers interact with each other to serve end customer needs. Training AI models is something that can be done on a schedule, modern web applications are load balanced all over the world, and site reliability engineering has brought about distributed architectures that are resilient to compute and connection loss at multiple scopes, including a full data center outage.
These facts play into the system such that a data center should be able to reduce power significantly to help utilities serve load for the few hours during a year when they need peak shaving and help avoid over building peaking capacity with fossil fuels.
Sam Altman is chairman of a Small Modular Nuclear company called OKLO though they haven't been approved by the regulators for their design. All analyses seem to say that SMRs are too expensive and take too long to build, and the power is too expensive . What do you think?
The internet is an abstract example of overthinking and over-chattering. A good portion of it consists of people trying to compete with one another for attention, which amounts to nothing. (Pardon the irony, but this very comment may serve as an example). Although the cleverness of the technology is impressive, it will eventually burn itself out for lack of justified purpose. The future is Amish.
Data shows that there are currently over 8,000 data centers worldwide, primarily located in the United States, Asia, and Europe. Northern Virginia in the United States is the world's largest data center hub, with around 300 data centers.
From an energy consumption perspective, data centers in Northern Virginia consumed 2,552 MW in 2023, while those in Dallas consumed 654 MW, Silicon Valley 615 MW, Beijing 1,799 MW, London 1,052 MW, and Frankfurt 864 MW.
In the global market, the biggest players are undoubtedly Amazon AWS, Microsoft Azure, and Google Cloud. They are continuously expanding and are indisputable international giants.
AI's demand for data centers is enormous, placing significant pressure on infrastructure. Operating data centers requires a substantial amount of electricity and water for cooling. The power consumption of hyper-scale data centers can be comparable to that of a city. New servers running GPT models consume 5-10 times more electricity. Consequently, to meet the development of AI, the industry needs to adjust data center designs and layouts, using better materials.
Due to the exponential growth of AI research, the demand for AI computing will significantly increase over the next decade.
Great post! But: one of your tables implies there are geographic requirements, such as proximity from airports, for Tier I/II, III, and IV data centers (Uptime Institute ratings). (
Your source was the data center builders Bible, but that resource seems to just provide recommended requirements, not Uptime Institute requirements.
I can’t seem to verify that claim. Seems like UI is just about infrastructure design and resulting theoretical uptime.
Why not build data centers where there is lots of power and cooling is less of an issue due to cold outdoor temperatures - like the James Bay area of Quebec?
Some of the requirements on a high level data center in the charts above included proximity to a major metro area and to an airport - probably this is because you want emergency technicians and backup parts to be able to get to the location quickly if anything bad happens. They also mentioned proximity to major internet hubs, so that latency of communications would be low.
If there's enough of an advantage, it might be possible to pay some people to live full-time near the datacenter to be on call, but unless there's some new construction of major internet nodes in these areas, that isn't going to be much of a benefit.
I could imagine Iceland becoming a major location though, with low temperatures, cheap electricity, location on some major transatlantic cables, and an established town where you could pay some technicians to live full time.
You read my mind. I'm looking into partnerships with Hydro Quebec, and the James Bay Cree, as well as looking at Montreal and Drommondville.
I'm curious how much the outdoor temperature really matters. I mean you still need active cooling in the summer and piping outside air deep into the structure would take a fair bit of space and engineering not to mention creating another potential point of failure. Also you can also use a large body of water and pipe in cold water to a heat exchanger (or maybe both aren't worth it).
Of course that only works for the training, you want servers relatively near the user.
Outdoor temperatures matter less, but there is excess power in Quebec. There has been a political battles for years over building transmission lines from Quebec south to New England. So why not put the data centers in Quebec and run fiber south which no one really cares about, unlike high voltage transmission lines.
You may want servers close to the users but latency may not be that much of an issue so a few more ms may not be a big deal if you have users in Boston-NY hitting a server in James Bay rather than Virginia.
Agreed. I'd like to use full immersion cooling with some kind of outdoor heat exchange to run a Stirling generator. Summer is still a problem, as is telecom.
I'm kinda surprised at the NIMBY issue. I mean a data center is in some sense the perfect neighbor as they will occupy high value real estate and have virtually no burden in terms of transit nor burden most city services.
For example, Quebec does not like data centers and cryptomines because they use a lot of electricity, provide few jobs, and because of tax policies the companies are almost always headquartered in Toronto or DE.
That makes sense, but it seems less like a NIMBY issue than an electricity and tax concern. I guess it's just a matter of terminology but that's not what I think of when I hear NIMBY.
HV transmission through residential areas is a major pushback. N Virginia growing from 3GW of datacenter power to 10 GW by 2035 is driving more 500kV routes needing new rights of way.
Trying to power 24 x 7 datacenters with intermittent renewables and battery storage, which would be primarily PV in N Virginia requiring tens of thousands of acres of farmland (several counties worth).
If you think NIMBY's object to HV transmission, wait until there are proposals for nuclear reactors in a residential neighborhood.
In the case of northern Quebec, running power lines and batteries through residential areas would be less of a problem, because the region is sparsely populated. And, of course, cold and with abundant hydropower.
The more lucrative a project is, and the more money at stake, the *more* sense it is to become a NIMBY and demand a toll/tax/"my fair share" for allowing the project to even exist, when I could block it and threaten the entire project. The company is willing to spend $10 billion on this new data center project? Good, that means I can demand $1 billion and not get laughed out of the room. The data center project isn't actually *that* objectionable? Who *cares* about that, it was never about the project itself, but the money I can extort from it. The project could be an orphanage for kittens and it'd *still* make sense for me to shake it down for cash, every single last cent it can give, as long as the law supports my NIMBY-ism.
Before leaving negative thoughts, I have to say I'm a big fan and love your posts! I devour all your posts with great interest ^^
But I really was not a fan of this one. It reads like an opinion piece or ad copy, and takes a very strong pro-USA and pro-AI stance. I guess your other posts aren't completely neutral on all topics, but I feel like they're much more evenhanded.
Anyway, happy you're getting writing gigs; I just hope similar ones won't be overly frequent in this feed in the future. :)
> In the long term, we may see immersion cooling, where the entire computer is immersed in a heat-conducting fluid.
Nitpick: All computers on Earth are immersed in a heat-conducting fluid. You probably mean "liquid". :)
We have not yet seen the real value of AI. So far it feels like a huge investment when we have much more pressing matters to attend to.
Climate change ? Education ? Health ?
AI as is has an enormous potential but will be used for foolish reasons and with no social integration at all. We only start to realize the huge impact social media and the digital world has on our social norms, our work and our culture…
AI is already taking jobs, by no means very interesting or exciting jobs, but still. How do we guarantee that AI won’t be just one more capitalist optimization of our society that leads rich to be richer and poor to be poorer and our environment to be a collateral damage AGAIN ? (Exaggerated but you get my point…)
We need a new social contract around AI usage, it’s crucial. As usual, not a skeptic regarding the technology but rather the use we make of it, seeing who’s leading that market I’m already shaking.
Something that would have been interesting to compare to was the crypto boom stupidity - another compute-intensive energy gobbling false prophet of the future.
love the focus on physical product side of things and hardware needed to sustain our digital infrastructure !
This is a nice piece! I am hopeful the series will include a post on the nature of how data centers interact with each other to serve end customer needs. Training AI models is something that can be done on a schedule, modern web applications are load balanced all over the world, and site reliability engineering has brought about distributed architectures that are resilient to compute and connection loss at multiple scopes, including a full data center outage.
These facts play into the system such that a data center should be able to reduce power significantly to help utilities serve load for the few hours during a year when they need peak shaving and help avoid over building peaking capacity with fossil fuels.
Great post
excellent post!
— danielle fong, lightcell energy
Sam Altman is chairman of a Small Modular Nuclear company called OKLO though they haven't been approved by the regulators for their design. All analyses seem to say that SMRs are too expensive and take too long to build, and the power is too expensive . What do you think?
I suspect the deciding factor for these are whether you can get a single approval for a design that gets exactly reproduced in multiple installations.
Also the US/EU aren't the only potential markets. China probably will want to go domestic but other countries might have less red tape.
Excellent writing!
Excellent information for organizations doing capital planning. Thanks!
The internet is an abstract example of overthinking and over-chattering. A good portion of it consists of people trying to compete with one another for attention, which amounts to nothing. (Pardon the irony, but this very comment may serve as an example). Although the cleverness of the technology is impressive, it will eventually burn itself out for lack of justified purpose. The future is Amish.
Data shows that there are currently over 8,000 data centers worldwide, primarily located in the United States, Asia, and Europe. Northern Virginia in the United States is the world's largest data center hub, with around 300 data centers.
From an energy consumption perspective, data centers in Northern Virginia consumed 2,552 MW in 2023, while those in Dallas consumed 654 MW, Silicon Valley 615 MW, Beijing 1,799 MW, London 1,052 MW, and Frankfurt 864 MW.
In the global market, the biggest players are undoubtedly Amazon AWS, Microsoft Azure, and Google Cloud. They are continuously expanding and are indisputable international giants.
AI's demand for data centers is enormous, placing significant pressure on infrastructure. Operating data centers requires a substantial amount of electricity and water for cooling. The power consumption of hyper-scale data centers can be comparable to that of a city. New servers running GPT models consume 5-10 times more electricity. Consequently, to meet the development of AI, the industry needs to adjust data center designs and layouts, using better materials.
Due to the exponential growth of AI research, the demand for AI computing will significantly increase over the next decade.
Great post! But: one of your tables implies there are geographic requirements, such as proximity from airports, for Tier I/II, III, and IV data centers (Uptime Institute ratings). (
Your source was the data center builders Bible, but that resource seems to just provide recommended requirements, not Uptime Institute requirements.
I can’t seem to verify that claim. Seems like UI is just about infrastructure design and resulting theoretical uptime.
What am I missing?