[cillian64]

One random preprint I found (https://arxiv.org/html/2505.09598v2) estimates inference is 90% of the total energy usage. From some googling around, everything seems to agree that inference dominates (or is at least comparable to) training on the large commonly-used models.

I was less surprised that inference dominates training after I read that chatgpt is serving billions of requests per day.