[ACCount36]

What? LLMs do benefit from economies of scale. There are a lot of things like MoE sharding or speculative decoding that only begin to make sense to set up and use when you're dealing with a large inference workload targeting a specific model. That's on top of all the usual datacenter economies of scale.

The whole thing with "OpenAI is bleeding money, they'll run out any day now" is pure copium. LLM inference is already profitable for every major provider. They just keep pouring money into infrastructure and R&D - because they expect to be able to build more and more capable systems, and sell more and more inference in the future.

[Yizahi]

Single LLM company can't stop investing into better systems and marketing of them, because there is no moat and customers will flee to the ones who do invest. It's free after all. So it is a closed loop which can't be broken, companies can but won't switch to "just inference". And with investing, all of the LLM companies are losing money a lot (on the LLMs specifically).