[aothms]

Consciousness is simply a very useful evolutionary trait (same as ability to fly or having sharp teeth) because it allows to better plan ahead and reflect and therefore survive - or by imprinting on us the fear of dying we can act to prevent it before having our offspring.

While consciousness is maybe more elusive in how it functions in a brain, it's a biological trait that makes not so much sense to compare to a LLM which simply regurgitates fragments from text produced by conscious humans. So I don't see an immediate conflict.

It would be interesting to include things like OpenWorm (and later OpenFish, OpenCat, OpenHuman) in the discussion, but decoupled from the biological mechanisms I find it hard to develop a stance on that.

[Aaron2222]

There's being intelligent, having the ability to think, plan, etc. But that doesn't explain how I feel that I'm here, how I'm conscious. But it could certainly explain why I say I am, being a consequence of how my brain functions. I know I am, because I can feel being conscious, but there's no reason for an AGI to believe that, other than it thinking it's conscious as well (so the same reason I'd believe that you are conscious).

[literalAardvark]

One can do planning ahead without being conscious, it can all be rules based.

Our high level consciousness seems like more of a side-effect, and I subscribe to [*my interpretation of] Peter Watts' Blindsight and Echopraxia that it's evolutionary dead weight that will be outcompeted by better adapted rules based organisms, likely from within our midst (high functioning psychopaths as the first adaptation).

Life doesn't have time for our navel gazing, art, computer games, reproduction avoidance, etc. It's wasted biological potential.

I highly recommend the books btw, Blindsight is entertaining hard sci-fi that's actually used as undergrad course support.