[naveen_k]

Good point. I'm often running multiple parallel jobs with varying priorities where uniform throttling actually makes sense. Many LLM inference tasks are long-running but not fully utilizing hardware (often waiting on I/O or running at partial capacity)

The dual Epyc CPUs (128 cores) in my setup have a relatively high idle power draw compared to consumer chips. Even when "idle" they're consuming significant power maintaining all those cores and I/O capabilities. By implementing uniform throttling when utilization is low, the automation actually reduces the baseline power consumption by a decent amount without much performance hit.

[foobarian]

It seems it may be relatively accessible to take a few representative tasks and actually measure the soup-to-nuts energy consumed at the plug. Would be very interesting to see in tandem with the power optimizations!

[naveen_k]

That's exactly what I did first! I ran a CPU torture test at full clock speed and measured the power draw at the plug, then repeated the same test with the lowest clock speed setting. For the Epyc system, there was about 225W lower power draw at the reduced clock speed. Even at idle, capping the max frequency reduced the power draw by about 20+%.