Being the best European AI company is also a multi billion business. Its not like China or the US respects GDPR. A lot of companies will choose the best European company.
The pre-training plateau is real. Nearly all the improvements since then have been around fine tuning and reinforcement learning, which can only get you so far. Without continued scaling in the base models, the hope of AGI is dead. You cannot reach AGI without making the pre-training model itself a whole lot better, with more or better data, both of which are in short supply.
Could it be that at least for the "lowest" fruits, most amazing things that can one can hope to obtain from scraping the whole web and throw it at some computation training was already achieved? Maybe AGI simply can not be obtained without some relevant additional probes sent in the wild to feed its learning loops?
LLMs haven't improved much. What's improved is the chat apps: switching between language model, vision, image and video generation and being able to search the internet is what has made them seem 100x more useful.
Run a single LLM without any tools... They're still pretty dumb.
Why would the debt matter when you have $60 billion in ad revenue and are generating $20 billion in op income? That's OpenAI 5-7 years from now, if they're able to maintain their position with consumers. Once they attach an ad product their margins will rapidly soar due to the comparatively low cost of the ad segment.
The technology is closer to a decade from seeing a plateau for the large general models. GPT o3 is significantly beyond o1 (much less 3.5 which was just Nov 2022). Claude 4 is significantly beyond 3.5. They're not subtle improvements. And most likely there will be a splintering of specialization that will see huge leaps outside the large general models. The radical leap in coding capabilities over the past 12-18 months is just an early example of how that will work, and it will affect every segment of human endeavour.
> Once they attach an ad product their margins will rapidly soar due to the comparatively low cost of the ad segment.
They're burning through computers and capital. No amount of advertising could cover the cost of training or even running these models. The massive subscription costs we've started seeing are just a small glimpse into the money they are burning through.
They will NOT make a profit using the current methods unless the models become at least 10 times more efficient than they are now. At which point can Europe adapt to the innovation without much cost.
It's an arms race to see who can burn the most money the fastest, while selling the result for as little as possible. When they need to start making money, it will all come crashing down.
sisve|8 months ago
ACCount36|8 months ago
People were claiming that since year 2022. Where's the plateau?
asadotzler|8 months ago
louiskottmann|8 months ago
psychoslave|8 months ago
dismalaf|8 months ago
LLMs haven't improved much. What's improved is the chat apps: switching between language model, vision, image and video generation and being able to search the internet is what has made them seem 100x more useful.
Run a single LLM without any tools... They're still pretty dumb.
adventured|8 months ago
The technology is closer to a decade from seeing a plateau for the large general models. GPT o3 is significantly beyond o1 (much less 3.5 which was just Nov 2022). Claude 4 is significantly beyond 3.5. They're not subtle improvements. And most likely there will be a splintering of specialization that will see huge leaps outside the large general models. The radical leap in coding capabilities over the past 12-18 months is just an early example of how that will work, and it will affect every segment of human endeavour.
aDyslecticCrow|8 months ago
They're burning through computers and capital. No amount of advertising could cover the cost of training or even running these models. The massive subscription costs we've started seeing are just a small glimpse into the money they are burning through.
They will NOT make a profit using the current methods unless the models become at least 10 times more efficient than they are now. At which point can Europe adapt to the innovation without much cost.
It's an arms race to see who can burn the most money the fastest, while selling the result for as little as possible. When they need to start making money, it will all come crashing down.
epolanski|8 months ago