[doe_eyes]

The jailbreaks are not doing anything worthwhile right now. They're fun to toy with and they give us insights into how LLMs work, but they don't unlock any superpowers. It's just a brand safety bypass, you can get the model to praise Hitler or something like that.

The way to make MDMA is easy to look up on the web. The government is not trying to keep a lid on it; instead, they're restricting access to key feedstock materials. In this case, safrole and isosafrole are on DEA List I, so no one is gonna sell it to you. If you ask an LLM for more novel and dangerous chemistry, you get plausibly-sounding nonsense, not superhuman AI.

Now, the hacks will become useful once models are given more agency, for example when fully automating customer support. But the existence of trivial bypasses is precisely what's holding these uses back.