The important part remains internalizing emission costs into the price of electricity. Fussing over individual users seems like a distraction to me. Rapid decarbonization of electricity is necessary regardless of who uses it. Demand will soar anyway as we electrify transportation, heating, and industry.
I agree but reducing consumption or increase of efficiency are still very important aspects of the energy transition. What is not consumed does not need to be generated.
Indeed. However the problem with LLMs is that vast amounts of VC money are being thrown at them, in the [misplaced] hope of great returns. This results in a resource mis-allocation of biblical proportions, of which unnecessary carbon emissions are a part.
If you are old enough you remember posting to Usenet and the warning that would accompany each new submission:
This program posts news to thousands of machines throughout the entire civilized world. Your message will cost the net hundreds if not thousands of dollars to send everywhere. Please be sure you know what you are doing. Are you absolutely sure that you want to do this? [ny]
Maybe we meed something similar in LLM clients. Could be phrased in terms of how many pounds of atmospheric carbon the request will produce.
> Tech companies like Meta, Amazon, and Google have responded to this fossil fuel issue by announcing goals to use more nuclear power. Those three have joined a pledge to triple the world’s nuclear capacity by 2025.
Erm ... that's a weird date considering this article came out yesterday. They actually pledge to triple the world's nuclear capacity by 2050[1]
There are a couple of weird things like that in this article, including the classic reference to "experts" for some of its data points. Still ... at least somebody's trying to quantify this.
> The largest model we tested has 405 billion parameters, but others, such as DeepSeek, have gone much further, with over 600 billion parameters.
Very quickly skimming, I have some trouble taking this post seriously when it omits that the larger DeepSeek one is a mixture-of-experts that will only use 12.5% (iirc) of its components for each token.
The best summary of text energy use I've seen is this (seemingly more rigorous, although its estimates are consistent with the final numbers made by the present post): epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use
Estimates for a given response widely for a "typical" query (0.3 Wh; 1080 joules) and a maximal-context query (40 Wh; 144k joules). Assuming most uses don't come close to maximizing the context, the energy use of text seems very small compared to the benefits. That being said, the energy use for video generation seems substantial
I would be interested in seeing the numbers corresponding to how LLMs are typically used for code generation
This series of articles is driving me insane. The authors or editors are using inappropriate units to shock readers: billions of gallons, millions of square feet. But they are not putting the figures into context that the reader can directly comprehend. Because if they said the Nevada data centers would use 2% as much water as the hay/alfalfa industry in Nevada then the entire article loses its shock value.
I agree. Even first few paragraphs strike me as intentionaly misleading. The whole "AI energy saga" seems to be full of manipulative claims. I can't help to feel that the intent is just to color the AI as bad in whatever way possible. It feels like it's deriving the research from predetermined conclusions
The number of people in this comment thread defending this gargantuan energy footprint for a technology that currently in large measure is being used for a tremendous amount of dogshit things (and oceans of visual/text spam) is amusing considering the hysterics that this same energy use problem caused when it came to crypto.
I guess it becomes okay when the companies guzzling the energy are some of the biggest tech employers in the world, buttering your bread in some way.
There's a certain irony here in the fact that this page is maxing out my CPU on idle, doing some unclear work in javascript, while I'm just reading text.
When companies make ESG claims, sensible measurement and open traceability should always be the first proof they must provide. Without these, and validation from a credible independent entity such as a non-profit or government agency, all ESG claims from companies are merely PR puff pieces to keep the public at bay (especially in "AI").
> In 2017, AI began to change everything. Data centers started getting built with energy-intensive hardware designed for AI, which led them to double their electricity consumption by 2023.
As we all know, the generative AI boom only really kicked into high gear in November 2022 with ChatGPT. That's five years of "AI" growth between 2017 and 2022 which presumably was mostly not generative AI.
I believe we're currently seeing AI in the "mainframe" era, much like the early days of computing, where a single machine occupied an entire room and consumed massive amounts of power, yet offered less compute than what now fits in a smartphone.
I expect rapid progress in both model efficiency and hardware specialization. Local inference on edge devices, using chips designed specifically for AI workloads, will drastically reduce energy consumption for the majority of tasks. This shift will free up large-scale compute resources to focus on truly complex scientific problems, which seems like a worthwhile goal to me.
The CPU development curve is often thrown around but it very seldomly fits anything else in reality. It was a very rare and extraordinary set of coincidences that got it us here. Computation using silicon turned out to have massive growth potential for a variety of lucky reasons but say battery tech is not so lucky, nor is fusion nor is quantum computing.
The low hanging fruit has been plucked by said silicon development process and while remarkable improvement in AI efficiency is likely it is highly unlikely for that to follow a similar curve.
More likely is slow, incremental process taking decades. We cannot just wish away billions of parameters and the need for trillions of operations. It’s not like we have some open path of possible improvement like with silicon. We walked that path already.
I don’t understand the “chips designed for AI workloads” sentiment I hear all the time. Llms were designed using Gpus. The hardware already exists, so what will make it use less energy in a world where Gpus over the last decade have only become bigger, hotter, more power hungry hardware? If we could develop Llm on anything less we probably would have shifted back to Cpus already.
It sure seems like that to me. I was pretty impressed by how easily I could run small Gemma on 7 year old laptop and get a decent chat experience.
I can imagine that doing some clever offloading to a normal programs and using the LLM as a sort of "fuzzy glue" for the rest could improve the efficiency on many common tasks.
What’s the net energy footprint of an employee working in an office whose job was made redundant by AI? Of course that human will likely have another job, but what’s the math of a person who was doing tedium solved by AI and now can do something more productive that AI can’t necessarily do. In other words, let’s calculate the “economic output per energy unit expended.”
On that note, what’s the energy footprint of the return to office initiatives that many companies have initiated?
> Of course that human will likely have another job, but what’s the math of a person who was doing tedium solved by AI and now can do something more productive that AI can’t necessarily do
That’s a lot of big assumptions - that the job getting replaced was tedious in the first place, that those other “more productive” job exists, that the thing AI can’t necessarily do will stay that way long enough for it not to be taken over by AI as well, that the tediousness was not part of the point (e.g. art)…
Net energy change of people doing work on their desk versus browsing the internet versus playing games, you will likely not see difference at all. They're all at rest, more or less thinking something. People at home sofa always have metabolic processes running regardless of whether it produces additional value to some corporation
I found this article to be a little too one sided. For instance, it didn’t talk about the 10x reductions in power achieved this past year — essentially how gpt4 can now run on a laptop.
Viz, via sama “The cost to use a given level of AI falls about 10x every 12 months, and lower prices lead to much more use. You can see this in the token cost from GPT-4 in early 2023 to GPT-4o in mid-2024, where the price per token dropped about 150x in that time period. Moore’s law changed the world at 2x every 18 months; this is unbelievably stronger.”
https://blog.samaltman.com/three-observations
I wouldn't be surprised if mankind will evolve similar to an organism and use 20% of all energy it produces on AI. Which is about 10x of what we use for software at the moment.
But then more AI also means more physical activity. When robots drive cars, we will have more cars driving around. When robots build houses, we will have more houses being built, etc. So energy usage will probably go up exponentially.
At the moment, the sun sends more energy to earth in an hour than humans use in a year. So the sun alone will be able to power this for the foreseeable future.
But the article says that energy use by AI is 48% more carbon intensive than the US average. So talk of solar power is a red herring -- that's not what it is running on now.
My go-to example is when some EU initiatives proposed labeling mobile phones by energy use. It completely missed the forest for the trees, as a prime example of overoptimization if your goal is carbon emissions reduction.
Nearly any other daily activity of a consumer in the developed world uses orders of magnitude more energy and resources than scrolling TikTok on a phone.
Examples?
– Driving to work: commuting burns far more fuel in a week than your phone uses in a year.
– Gym sessions: heated, lit, air-conditioned spaces plus transit add up quickly.
– Gaming or watching TV: bigger screens, bigger compute easily 100x and higher power needs vs phone gaming.
– Casually cooking at home: using a metric ton of appliances (oven, stove, fridge, pans) powered like twice a week, replaced every ~10 years.
– Reading print media: a daily newspaper or weekly book involves pulp, ink, shipping, and disposal.
– Streaming on a laptop or smart TV: even this draws more power than your phone.
– Taking a shower: the hot water energy use alone dwarfs your daily phone charge.
Of couse not doing any sports or culture is also not what societies want, but energy wise a sedentary passive tiktok lifestyle is as eco friendly as it get's vs. any other real world example.
Phones are basically the least resource-intensive tool we use regularly.
Externalities, context, and limited human time effects matter a lot more than what one phone uses vs the other.
Even e-readers already break even with books after 36 paper equivalents
If you don't want to go there, it doesn't really matter how much energy the human uses because the human will just use the same energy to do something else.
I've been thinking about this. If the human-equivalent of training an LLM is sending hundreds/thousands of students through college for many years, I can't help but think that the energy needed for both outcomes is comparable.
I'm worried about the environmental impacts of this, but from everything I've seen society values model output more. Curious to watch this over the rest of the decade.
[+] [-] BewareTheYiga|10 months ago|reply
[+] [-] adrianN|10 months ago|reply
[+] [-] seb1204|10 months ago|reply
[+] [-] hermitShell|10 months ago|reply
Making electricity so abundant and efficient is probably more solvable. You can’t solve stu… society
[+] [-] unknown|10 months ago|reply
[deleted]
[+] [-] somewhereoutth|10 months ago|reply
[+] [-] unknown|10 months ago|reply
[deleted]
[+] [-] unknown|10 months ago|reply
[deleted]
[+] [-] designerarvid|10 months ago|reply
[+] [-] whimsicalism|10 months ago|reply
[+] [-] SoftTalker|10 months ago|reply
This program posts news to thousands of machines throughout the entire civilized world. Your message will cost the net hundreds if not thousands of dollars to send everywhere. Please be sure you know what you are doing. Are you absolutely sure that you want to do this? [ny]
Maybe we meed something similar in LLM clients. Could be phrased in terms of how many pounds of atmospheric carbon the request will produce.
[+] [-] troyvit|10 months ago|reply
Erm ... that's a weird date considering this article came out yesterday. They actually pledge to triple the world's nuclear capacity by 2050[1]
There are a couple of weird things like that in this article, including the classic reference to "experts" for some of its data points. Still ... at least somebody's trying to quantify this.
[1] https://www.world-nuclear-news.org/articles/amazon-google-me...
[+] [-] bogtog|10 months ago|reply
Very quickly skimming, I have some trouble taking this post seriously when it omits that the larger DeepSeek one is a mixture-of-experts that will only use 12.5% (iirc) of its components for each token.
The best summary of text energy use I've seen is this (seemingly more rigorous, although its estimates are consistent with the final numbers made by the present post): epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use
Estimates for a given response widely for a "typical" query (0.3 Wh; 1080 joules) and a maximal-context query (40 Wh; 144k joules). Assuming most uses don't come close to maximizing the context, the energy use of text seems very small compared to the benefits. That being said, the energy use for video generation seems substantial
I would be interested in seeing the numbers corresponding to how LLMs are typically used for code generation
[+] [-] jeffbee|10 months ago|reply
[+] [-] panstromek|10 months ago|reply
[+] [-] southernplaces7|10 months ago|reply
I guess it becomes okay when the companies guzzling the energy are some of the biggest tech employers in the world, buttering your bread in some way.
[+] [-] neves|10 months ago|reply
Impressive how Big Tech refuses to share data with society for collective decisions.
I'd also recommend the Data Vampires podcast series:
https://techwontsave.us/episode/241_data_vampires_going_hype...
https://techwontsave.us/episode/243_data_vampires_opposing_d...
https://techwontsave.us/episode/245_data_vampires_sacrificin...
https://techwontsave.us/episode/247_data_vampires_fighting_f...
[+] [-] panstromek|10 months ago|reply
[+] [-] guappa|10 months ago|reply
[+] [-] vanschelven|10 months ago|reply
(cgroups, as per a sibbling comment, are addressed in this write-up as "not maximally satisfying")
[+] [-] mentalgear|10 months ago|reply
[+] [-] stevage|10 months ago|reply
[+] [-] kkarakk|10 months ago|reply
[+] [-] simonw|10 months ago|reply
> In 2017, AI began to change everything. Data centers started getting built with energy-intensive hardware designed for AI, which led them to double their electricity consumption by 2023.
As we all know, the generative AI boom only really kicked into high gear in November 2022 with ChatGPT. That's five years of "AI" growth between 2017 and 2022 which presumably was mostly not generative AI.
[+] [-] xeox538|10 months ago|reply
I expect rapid progress in both model efficiency and hardware specialization. Local inference on edge devices, using chips designed specifically for AI workloads, will drastically reduce energy consumption for the majority of tasks. This shift will free up large-scale compute resources to focus on truly complex scientific problems, which seems like a worthwhile goal to me.
[+] [-] whatnow37373|10 months ago|reply
The low hanging fruit has been plucked by said silicon development process and while remarkable improvement in AI efficiency is likely it is highly unlikely for that to follow a similar curve.
More likely is slow, incremental process taking decades. We cannot just wish away billions of parameters and the need for trillions of operations. It’s not like we have some open path of possible improvement like with silicon. We walked that path already.
Maybe photonics..
[+] [-] righthand|10 months ago|reply
[+] [-] panstromek|10 months ago|reply
I can imagine that doing some clever offloading to a normal programs and using the LLM as a sort of "fuzzy glue" for the rest could improve the efficiency on many common tasks.
[+] [-] briandear|10 months ago|reply
On that note, what’s the energy footprint of the return to office initiatives that many companies have initiated?
[+] [-] lm28469|10 months ago|reply
Like driving a uber or delivering food on a bicycle ? Amazing!
[+] [-] folkrav|10 months ago|reply
That’s a lot of big assumptions - that the job getting replaced was tedious in the first place, that those other “more productive” job exists, that the thing AI can’t necessarily do will stay that way long enough for it not to be taken over by AI as well, that the tediousness was not part of the point (e.g. art)…
[+] [-] Scarblac|10 months ago|reply
[+] [-] carunenjoyerlp|10 months ago|reply
[+] [-] dr_dshiv|10 months ago|reply
I found this article to be a little too one sided. For instance, it didn’t talk about the 10x reductions in power achieved this past year — essentially how gpt4 can now run on a laptop.
Viz, via sama “The cost to use a given level of AI falls about 10x every 12 months, and lower prices lead to much more use. You can see this in the token cost from GPT-4 in early 2023 to GPT-4o in mid-2024, where the price per token dropped about 150x in that time period. Moore’s law changed the world at 2x every 18 months; this is unbelievably stronger.” https://blog.samaltman.com/three-observations
[+] [-] teekert|10 months ago|reply
[+] [-] amelius|10 months ago|reply
[+] [-] mg|10 months ago|reply
I wouldn't be surprised if mankind will evolve similar to an organism and use 20% of all energy it produces on AI. Which is about 10x of what we use for software at the moment.
But then more AI also means more physical activity. When robots drive cars, we will have more cars driving around. When robots build houses, we will have more houses being built, etc. So energy usage will probably go up exponentially.
At the moment, the sun sends more energy to earth in an hour than humans use in a year. So the sun alone will be able to power this for the foreseeable future.
[+] [-] Scarblac|10 months ago|reply
[+] [-] janaagaard|10 months ago|reply
- "by 2028 [...] AI alone could consume as much electricity annually as 22% of all US households."
What would the 22% be if compared against all US energy instead of just all US household?
[+] [-] EdiX|10 months ago|reply
[1] https://rpsc.energy.gov/energy-data-facts
[+] [-] thedevilslawyer|10 months ago|reply
[+] [-] mxfh|10 months ago|reply
Nearly any other daily activity of a consumer in the developed world uses orders of magnitude more energy and resources than scrolling TikTok on a phone.
Examples?
– Driving to work: commuting burns far more fuel in a week than your phone uses in a year.
– Gym sessions: heated, lit, air-conditioned spaces plus transit add up quickly.
– Gaming or watching TV: bigger screens, bigger compute easily 100x and higher power needs vs phone gaming.
– Casually cooking at home: using a metric ton of appliances (oven, stove, fridge, pans) powered like twice a week, replaced every ~10 years.
– Reading print media: a daily newspaper or weekly book involves pulp, ink, shipping, and disposal.
– Streaming on a laptop or smart TV: even this draws more power than your phone.
– Taking a shower: the hot water energy use alone dwarfs your daily phone charge.
Of couse not doing any sports or culture is also not what societies want, but energy wise a sedentary passive tiktok lifestyle is as eco friendly as it get's vs. any other real world example.
Phones are basically the least resource-intensive tool we use regularly. Externalities, context, and limited human time effects matter a lot more than what one phone uses vs the other.
Even e-readers already break even with books after 36 paper equivalents
https://www.npr.org/2024/05/25/1252930557/book-e-reader-kind...
[+] [-] ahtihn|10 months ago|reply
If you don't want to go there, it doesn't really matter how much energy the human uses because the human will just use the same energy to do something else.
[+] [-] wyre|10 months ago|reply
[+] [-] coolcase|10 months ago|reply
[+] [-] Ericson2314|10 months ago|reply
[+] [-] lucb1e|10 months ago|reply
A few big consumers in centralized locations isn't changing the grid as much as the energy transition from fuels to electricity is
[+] [-] pizzuh|10 months ago|reply
I'm worried about the environmental impacts of this, but from everything I've seen society values model output more. Curious to watch this over the rest of the decade.
[+] [-] jes5199|10 months ago|reply