Let me save you 20 minutes of your life: This guy's lead argument is that the smart people who advise caution in creating super-intelligence look weird. He's quite proud of this argument.
I'll expand for those asking. The first third of the presentation does such an excellent job of steel-manning the case for concern, that I couldn't wait to hear his arguments against it. When he gets around to that, he reaches for a made-up concept he calls the "outside view" where he argues you should ignore rational arguments (aka the "inside view") if the person making those rational arguments seems weird.* Slides follow showing VCs in bad PR photos. What more evidence does anyone need?
*"But the outside view tells you something different. ... Even though their arguments are irrefutable, everything in your experience tells you you're dealing with a cult. Of course, they have a brilliant argument for why you should ignore those instincts, but that's the inside view talking."
He's right. Predictions about the future aren't actually more accurate because they have hundreds of pages of probability theory. Cults look convincing from inside the cult.
One should note that the AI people thought AI would look completely different than an LLM does (they thought it'd be an agent using a manually coded knowledge graph) but are still applying all their same arguments to this very different thing.
They do look weird to a lot of us. In excel, its trivial to fit a smoothed bendy line between points on a scatter plot. Because of the underlying math (taylor series) when you reach the end of your data points the line flings off into positive or negative INF.
AI super intelligence arguments have the same logic: the data ends so the interpolation now goes to infinity. This is such an obvious counter and these people know this math.
mode80|2 years ago
*"But the outside view tells you something different. ... Even though their arguments are irrefutable, everything in your experience tells you you're dealing with a cult. Of course, they have a brilliant argument for why you should ignore those instincts, but that's the inside view talking."
astrange|2 years ago
One should note that the AI people thought AI would look completely different than an LLM does (they thought it'd be an agent using a manually coded knowledge graph) but are still applying all their same arguments to this very different thing.
mjklin|2 years ago
ImaCake|2 years ago
AI super intelligence arguments have the same logic: the data ends so the interpolation now goes to infinity. This is such an obvious counter and these people know this math.
gunshai|2 years ago
Extrapolation not interpolation, proceed with downvotes.
jodrellblank|2 years ago
coldtea|2 years ago