[totoglazer]

That energy is still being consumed if it’s happening on device. It’s just more spread out, and potentially less efficient.

[bugglebeetle]

The rate of efficiency optimizations that have occurred in the open source community over the past two years I think call that into question. Services like OpenAI and Anthropic have the highest performing models, but since we have no ideas about what they really are or how they do inference, we can’t say that they’re necessarily more efficient than open source. In fact, people doing things in walled gardens, motivated by maximizing market share, subsidized by big players, are more likely to be doing things inefficiently than research being done out in the open.