[frotaur]

It astonishes me how people can make categorical judgements on things as hard to define as 'understanding'.

I would say that, except for the observable and testable performance, what else can you say about understanding?

It is a fact that LLMs are getting better at many tasks. From their performance, they seem to have an understanding of say python.

The mechanistic way this understanding arises is different than humans.

How can you say then it is 'not real', without invoking the hard problem of consciousness, at which point, we've hit a completely open question.

[matthewkayin]

I think it is fair to say that AIs do not yet "understand" what they say or what we ask them.

When I ask it to use a specific MCP to complete a certain task, and it proceeds to not use that MCP, this indicates a clear lack of understanding.

You might say that the fault was mine, that I didn't setup or initialize the MCP tool properly, but wouldn't an understanding AI recognize that it didn't have access to the MCP and tell me that it cannot satisfy my request, rather than blindly carrying on without it?

LLMs consistently prove that they lack the ability to evaluate statements for truth. They lack, as well, an awareness of their unknowing, because they are not trying to understand; their job is to generate (to hallucinate).

It astonishes me that people can be so blind to this weakness of the tool. And when we raise concerns, people always say

"How can you define what 'thinking' is?" "How can you define 'understanding'?"

These philosophical questions are missing the point. When we say it doesn't "understand", we mean that it doesn't do what we ask. It isn't reliable. It isn't as useful to us as perhaps it has been to you.

[phantasmish]

To be fair, it can be hard to define “chair” to the satisfaction of an unsympathetic judge.

“Do chairs exist?”:

https://m.youtube.com/watch?v=fXW-QjBsruE

[encyclopedism]

You can make categorical judgements on many things that are hard to define. Often by taking the opposite approach. It's hard to define what consciousness itself is yet I can define what it isn't and that teases a lot out.

So I can categorically say LLM's do not understand by quite literally understanding what NOT understanding is.

We know what LLM's are and what they are NOT.

Please see my earlier comment above:

> LLM's do not think, understand, reason, reflect, comprehend and they never shall.

[kunley]

How do you know what kind of "understanding" a python has? Why python and not a lizard? Or a bird? What method do you use for evaluating this? Does a typical python do what you tell your ai agent to do?...

C'mon, this comparison seems to be very, very unscientific. No offense...

[edanm]

I'm pretty sure parent poster meant they understand Python, the programming language, not that they have the understanding of the animal python.