[orbital-decay]

This type of research requires experimentation (mostly failures) on extremely complex real-world equipment. Same with the nuclear weapons. AI being able to magically figure it out without experimental grounding is pure and absolute fantasy, used by companies like OpenAI and Anthropic as a justification for monopolizing AI R&D. In a sense it's not surprising this idea comes from rationalism-adjacent folks, as rationalism is mostly about the idea that experimentation is irrelevant and you can infer anything using just logic alone.

[dennis_jeeves2]

>rationalism is mostly about the idea that experimentation is irrelevant and you can infer anything using just logic alone.

Thanks for putting it the way you did. I didn't knew it was meant be that way, but it sort of confirms my suspicion that people who use the term 'rational' and 'logic' loosely often to dismiss an opposing view never really seek experimental results before having a point of view.

[totetsu]

Okay, I’m inclined to agree there.. but I can’t find the reference now, I also read that the worry is that the complexity required is coming down fast, and while maybe it’s not going to happen in a basement , there could be a small scale lab, not subject to rigorous certifications and checks that offers crisper as a service, and ai could be used to .. just perturb a protein a little bit so it doesn’t trigger some known virus black list, and people could just be ordering things online.

[KPGv2]

> In a sense it's not surprising this idea comes from rationalism-adjacent folks, as rationalism is mostly about the idea that experimentation is irrelevant and you can infer anything using just logic alone.

Yeah IIRC Yudkowski famously said something about a super intelligence could derive the theory of gravity correctly by seeing only three frames of a video depicting an apple falling from a tree. This is the same Less Wrong nonsense, rejecting how vital and irreplaceable experimentation is.

There's an infinite number of explanations for the location of an object in three equally time-spaced instances. Not to mention limitations of the measuring equipment itself.