[dahart]

It might be a mistake to assume tomorrow’s training looks like today’s. Unsupervised learning is a thing and a very hot research topic, precisely because it avoids some of today’s big problems with acquiring the vast amounts of training data necessary.

[greentea23]

Unsupervised leanring has been around for years and is already how the current wave of models are trained. It doesn't mean no data, it means no human provided labels of the data. So you still need creative new human ideas to move LLMs forward. LLMs != intelligence.

[varispeed]

Exactly this. Even frontier models like Opus 4.6 have absolutely zero understanding of the task at hand. If you give them problem they have not encountered in training data, they will not solve it. You can however, guide them to resolve the problem, but in that case these get reduced to merely an auto complete. Don't get me wrong - models are getting better and hide very well that they don't understand anything, you can almost get fooled now.