[mo_42]

Humanity has bootstrapped itself out of a lot of BS over the centuries. There's a mechanism for discarding bad ideas. For example:

Badly-designed boats just don't return.

Ill-designed protection of cities means they'll be conquered.

Scientific ideas that do not corroborate, will be discarded.

etc.

Our current approach to AI doesn't have this mechanism. In the past, humanity just implemented ideas: a city was built according to some weird idea and lasted centuries. So the original idea would spread and be refined by further generations. I guess we need to bring such a mechanism into the loop.