[mike_hearn]

This meme needs to go in the bin. Loss making companies love inventing strange new accounting metrics, which is one reason public companies are forced to report in standardized ways.

There's no such thing as "profitable inference". A company is either profitable or it isn't.

Let's for a second assume all the labs somehow manage to form a secret OPEC-style cartel that agrees to slow training to a halt, and nobody notices or investigates. This is already hard to imagine with the amount of scrutiny they're under and given that China views this as a military priority. But let's pretend they manage it. These firms also have lots of other costs:

• Staffing and comp! That's huge!

• User subsidies to allow flat rate plans

• Support (including abuse control and handling the escalations from their support bots)

• Marketing

• Legal fees and data licensing

• Corporate/enterprise sales, which is expensive as hell even though it's often worth it

• Debt servicing (!!)

• Generating returns for investors

Inferencing margins have to cover all of those, even if progress stops tomorrow and the RoI to investors has to be likewise very large, so margins can't be trivial. Yet what these firms have said about their margins is very ambiguous. As they're arriving at this statement by excluding major cost components like training, it's not clear what they think the cost of inferencing actually is. Are they excluding other things too like hw depreciation and upgrades? Are they excluding the cost of the corporate sales/support infrastructure around the inferencing?

[rodonn]

"profitable on inference" means "marginal costs of inference are lower than revenue". It is very common to distinguish between upfront costs vs. marginal costs when judging the economic viability of a business.

You mention "debt servicing", but OpenAI has no debt. All the money they have raised is equity not debt.

[tibbar]

To be clear, it's absolutely impossible for OpenAI and the others to stop. The valuation and honestly the global markets depend on them staying leveraged to the hilt. So they're not going to stop. However, the point is that the models are genuinely useful and people pay for them, and if we reset the timeline with a company that has just the current proprietary models, they could turn a profit. That might involve charging more than they do now, etc. But this is much different than OpenAI, specifically, trying to turn a profit today, which wouldn't work for many reasons.

But also, "profitable inference" IS a thing! "Gross margin" is important and meaningful, even if a company has other obligations that mean it's overall not profitable.