[someexgamedev]

This is how I feel about AGI too, and I also include self-driving cars. I don't think those are just around the corner either.

In general I don't think our current approach to AI is all that clever. It brute forces algorithms which no human has any comprehension of or ability to modify. All a human can do is modify the input data set and hope a better algorithm (which they also don't understand) arises from the neural network.

It's like a very permissive compiler which produces a binary full of runtime errors. You have to find bugs at runtime and fiddle with the input until the runtime error goes away. Was it a bug in your input? Or a bug in the compiler? Who knows. Change whichever you think of first. It's barely science and it's barely a debug workflow.

What pushed me all the way over the edge was when adversarial techniques started to be applied to self-driving cars. That white paper made them look like death machines. This entire development process I am criticising assumes we get to live in the happy path, and we're not. The same dark forces infosec can barely keep at bay on the internet, and have completely failed to stop on IoT, will now be able to target your car as well.

Worst thing is all our otherwise brilliant humans like Carmack are gonna be the guinea pigs in the cars as they head off toward their next runtime crash.