[phforms]

Maybe the structure and operation in LLMs is a somewhat accurate model of the structure and operation of our brains and maybe the actual representation of “thought” is different between the human brain and LLMs. Then it might be the case that what makes the LLM “feel human” depends not so much on the actual thinking stuff but how that stuff is related and how this process of thought unfolds.

I personally believe that our thinking is fundamentally grounded/embodied in abstract/generalized representations of our actions and experiences. These representations are diagrammatic in nature, because only diagrams allow us to act on general objects in (almost) the same way to how we act on real-world objects. With “diagrams” I mean not necessarily visual or static artefacts, they can be much more elusive, kinaesthetic and dynamic. Sometimes I am conscious of them when I think, sometimes they are more “hidden” underneath a symbolic/language layer.