[charleshn]

I'm always surprised by the number of people posting here that are dismissive of AI and the obvious unstoppable progress.

Just looking at what happened with chess, go, strategy games, protein folding etc, it's obvious that pretty much any field/problem that can be formalised and cheaply verified - e.g. mathematics, algorithms etc - will be solved, and that it's only a matter of time before we have domain-specific ASI.

I strongly encourage everyone to read about the bitter lesson [0] and verifier's law [1].

[0] http://www.incompleteideas.net/IncIdeas/BitterLesson.html

[1] https://www.jasonwei.net/blog/asymmetry-of-verification-and-...

[mvieira38]

Your examples are not LLMs, though, and don't really behave like them at all. If we take the chess analogy and design an "LLM-like chess engine", it would behave like an average 1400 London spammer, not like Stockfish, because it would try to play like the average human plays in it's database.

It isn't entirely clear what problem LLMs are solving and what they are optimizing towards... They sound humanlike and give some good solutions to stuff, but there are so many glaring holes. How are we so many years and billions of dollars in and I can't reliably play a coherent game of chess with ChatGPT, let alone have it be useful?

[charcircuit]

>because it would try to play like the average human plays in it's database.

Why would it play like the average? LLMs pick tokens to try and maximize a reward function, they don't just pick the most common word from the training data set.

[throw310822]

Maybe you didn't realise that LLMs have just wiped out entire class of problems, maybe entire disciplines- do you remember "natural language processing"? What, ehm, happened to it?

Sometimes I have the feeling that what happened with LLMs is so enormous that many researches and philosophers still haven't had time to gather their thoughts and process it.

I mean, shall we have a nice discussion about the possibility of "philosophical zombies"? On whether the Chinese room understands or not? Or maybe on the feasibility of the mythical Turing test? There's half a century or more of philosophical questions and scenarios that are not theory anymore, maybe they're not even questions anymore- and almost from one day to the other.

[bigyabai]

People assume (rightly so) that the progress in AI should be self-evident. If the whole thing is really working that great, we should expect to see real advances in these fields. Protein-folding AI should lower the prices of drugs and create competitive new treatments at an unprecedented rate. Photo and video AI should be enabling film directors and game directors to release higher-quality content faster than ever before. Text AI should be spitting out Shakespeare-toppling opuses on a monthly basis.

So... where's the kaboom? Where's the giant, earth-shattering kaboom? There are solid applications for AI in computer vision and sentiment analysis right now, but even these are fallible and have limited effectiveness when you do deploy them. The grander ambitions, even for pared-back "ASI" definitions, is just kicking the can further down the road.

[TheBicPen]

The kaboom already happened on user-generated media platforms. YouTube, Facebook, tiktok, and so on are flooded with AI-generated videos, photos, sounds, and so on. The sheer volume of this low-quality slop is because AI lowered the barrier of entry for creating content. In this space the progress is not happening through pushing the upper bound of quality higher but by reducing the cost for minimal quality to down to near-0.

[tim333]

It's like the web in 1998. Everyone could see the potential but nothing actually worked very well.

[bwfan123]

> I'm always surprised by the number of people posting here that are dismissive of AI and the obvious unstoppable progress

Many of us have been through previous hype-cycles like the dot-com boom, and have learned to be skeptical. Some of that learning has been "reinforced" by layoffs in the ensuing bust (reinforcement learning). A few claims in your note like "it's only a matter of time before we have domain-specific ASI" are jarring - as you are "assuming the sale". LLMs are great as a tool for some usecases - nobody denies that.

The investment dollars are creating a class of people who are fed by those dollars, and have the incentive to push the agenda. The skeptics in contrast have no ax to grind.

[overgard]

We need to stop calling what we have AI. LLMs can't reliably reason. Until they can the progress is far from unstoppable.

[kadushka]

I love it how people are transitioning from “LLMs can’t reason” to “LLMs can’t reliably reason”.

[oytis]

It's very different from chess etc. If we could formalise and "solve" software engineering precisely, it would be really cool, and probably indeed just lift programming to a new level of abstraction.

I don't mind if software jobs move from writing software to verifying software either if it makes the whole process more efficient and the software becomes better as a result. Again, not what is happening here.

What is happening, at least in AI optimist CEO minds is "disruption". Drop the quality while cutting costs dramatically.

[charleshn]

I mentioned algorithms, not software engineering, precisely for that reason.

But the next step is obviously increased formalism via formal methods, deterministic simulators etc, basically so that one could define an environment for a RL agent.

[yeasku]

Open AI failed to create a bot that can play Dota. Llms cant learn complex heuristics.

[tim333]

I'm surprised too. You'd think tech people would understand what's going on. But of the prior replies to your comment 7 out of 8 seem dismissive. I'm in the "obvious unstoppable progress" camp but we seem to be a minority.

I guess maybe it isn't that obvious - I've read quite a lot in the area. People saying LLMs aren't very good are a bit like people long ago saying chess programs aren't very good. It was true but there was an inevitable advance as the hardware got better and then that led to enthusiasm to improve the software and computers became better than humans in a rather predictable way. It's driven in the end by hardware improvements. Whether the software is LLM or some other algo is kind of unimportant.

[Tainnor]

Mathematics cannot be "solved", that's a consequence of Gödel's First Incompleteness Theorem.

It can already be "cheaply verified" in the sense that if you write a proof in, say, Lean, the compiler will tell if you if it's valid. The hard part is coming up with the proof.

It may be possible that some sort of AI at some stage becomes as good, or even better than, research mathematicians in coming up with novel proofs. But so far it doesn't look like it - LLMs seem to be able to help a little bit with finding theorems (e.g. stuff like https://leansearch.net/), but to my understanding they are rather poor beyond that.

[charleshn]

You can now add getting gold at IMO [0] to the above list.

[0] https://x.com/alexwei_/status/1946477742855532918

[bwfan123]

on the surface this is a great achievement - if it holds . alpha-geometry required 1) human formalization of the question and 2) a solver for geometry

If the questions were given as-is (without a human formalizing it) and the llm didnt need domain solvers, and the llm was not trained on it already (which happened with frontier math) - I would be impressed.

Based on the past history with frontier math [1][2] I remain skeptical. The skeptic in me says that this happens prior to big announcements (GPT-5) to create the hype.

Finally, this article shows that LLMs were just bluffing in the usamo 2025 [3].

[1] https://www.reddit.com/r/slatestarcodex/comments/1i53ih7/fro...

[2] https://x.com/DimitrisPapail/status/1888325914603516214

[3] https://arxiv.org/pdf/2503.21934

[rcpt]

Have you ever seen a company say "welp, we wrote all the code. Now we're done?"