[bubblelicious]

Where does this view come from? I’m not aware of any real evidence for this. Also consider our data center buildouts in 26 and 27 will be absolutely extraordinary, and scaling is only at the beginning. You have a growing flywheel and plenty of synthetic data to break the data wall

[ModernMech]

Let me put it this way: when ChatGPT tells me I've hit the "Free plan limit for GPT-5", I don't even notice a difference when it goes away or when it comes back. There's no incentive for me to pay them for access to 5 if the downgraded models are just as good. That's a huge problem for them.

[riffraff]

Ditto for Gemini Pro and Flash, which I have on my phone.

I've been traveling in a country where I don't speak the language and or know the customs, and I found LLMs useful.

But I see almost zero difference between paid and unpaid plans, and I doubt I'd pay much or often for this privilege.

[bubblelicious]

This based on any non anecdotal evidence by chance?

[_aavaa_]

It is a problem easily solved with advertising.

[candiddevmike]

We need a fundamental paradigm shift beyond transformers. Throwing more compute or data at it isn't pushing the needle.

[marcosdumay]

Just to point, but there's no more data.

LLMs would always bottleneck on one of those two, as computing demand grows crazy quickly with the data amount, and data is necessarily limited. Turns out people threw crazy amounts of compute into it, so the we got the other limit.

[bubblelicious]

And you don’t think that’s already happening? Also where is your evidence for this?

[skywhopper]

There is zero evidence that synthetic data will provide any real benefit. All common sense says it can only reinforce and amplify the existing problems with LLMs and other generative “AI”.

[bubblelicious]

Sounds like someone has no knowledge of the literature, synthetic data isn’t like asking ChatGPT to give you a bunch of fake internet data.