[shurcooL]

I wanted to find some discussion on this topic. I’m not pessimistic, but I am curious to consider what it may mean for humanity if/when AGI happens.

[patentatt]

My pet theory is that anything we regard as truly intelligent has to have a sense of self, an identity if you will. In addition, it has to be motivated by something. All living things, and all the intelligent living things, have a survival and reproduction instinct on some level which is an animating force driving higher level thinking and actions. Basically, any AGI has to have something approaching a ‘soul’ to be ‘intelligent.’ And if that emerges, combined with exponential iterative evolution that isn’t limited by biology ... well all the sci-fi tropes seem plausible.