[seiji]

strong AI can be immortal

My grandfather was very racist and never changed as long as he lived. I'm glad most people aren't immortal. There's no guarantee a full AI wouldn't be tempted by evil and just become republican (or worse, a VC).

It can also recursively improve itself.

So can people (the more you know, the more you can learn), but most don't. It's important to remember "intelligence" isn't an abstract concept—intelligence is also embodied in personality—and personalities have wishes and goals and desires and loves and hates and that one song they can't get out of their head. A true "strong AI" will be fully conscious, not just algorithmic function bating.

Good luck telling a mildly strong godform to stop tripping on youtube videos and instead solve the global economic stability equation over lunch.

This advantage will lower the bar for strong AI even more.

That's kinda foofy conjecture. Being good at rectangular grid outcomes isn't necessarily a step in any direction towards a hands-off tax evaluating robot.

It feels really really good to talk about how AI will be a hundred billion trillion times smarter than the combined brainpower of all humans that have ever lived, but it feels good in the same way thinking dead people live again after they die feels good—it triggers that warm wishful thinking parietal lobe that removes a bit of reason for the sake of an overarching calmness.

Enthusiasm is great, but tempering with real expectations and less technopriesthood is better.

[partycoder]

You can recruit neurons within your own brain, not plug the equivalent of 4 brains into your brain and increase your cognitive abilities.

Now, while many people die and their ideas die with them, there are built in aspects of our brain that are hardwired. People eat animals because they are tasty, people follow the life of Kim Kardashian and waste money buying a rolex and wear fur just because the stupid way our social functions are hardwired in our brain.

[quietplatypus]

Pretty much this.

[nzonbi]

In the grand scheme of things, what is the point of humans repeating billion of times the same cycle?: born, grow, learn, achieve some things, have some fun, and die. It is awesome a few times. But it is worth to repeat and repeat that for ever? Not if there is a much better option: The singularity. It has the potential to enhance the human race in formidable ways. Yes,there is the risk that the AI may chose to destroy us. But in my opinion is a bet that we should take. Just try our best to not be destroyed in the way. But we shouldn't leave this opportunity unexplored. It would be the biggest achievement of humanity.

[hutzlibu]

It is NOT the same cycle. Every life is different. So what the people learned 500 years ago made sense for them - but they would face a hard time now. Our knowledge of daily living will be mostly obsolete in 200 (or much less) years, too.

Fresh minds don't have the burden of outdated ideas... quoting Max Planck: "a new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it."

[armitron]

You are assigning value to "us" beyond what might actually be there. What if our only evolutionary value is to give rise to machine intelligence [that will go on to explore the universe] and die off?

We are primitive forms of life after all and are at the point where we should be able to say, with certainty, that machine intelligence will be objectively superior and something that we should feel OBLIGATED to give rise to, regardless of consequences.

[tim333]

Yeah, the dying thing's a bitch.