Looks like the actual threat is that it's hard to get currently known chemical weapons synthesized because labs will refuse to do so, while it could be much easier to have some novel AI-generated molecule synthesized because the labs don't know what it does.
Seems easily countered by using the same toxicity prediction software when evaluating synthesis requests (but I'm not sure whether this actually matters, or whether skilled chemists can easily synthesize anything themselves anyway).
That's part of it. There's also the risk of clandestine operators discovering easily made chemical weapons that do not require a professional lab to create them in bulk. A nation state, or well funded terrorist group could exploit such a thing without too much effort.
It's important to remember. Chemical warfare can be used for mass destruction, but this approach could be used for other nefarious things at smaller more discrete scales en masse. ie 1000 attacks with different agents in each one... Forensic nightmare.
I like your suggestion to counter these things, but, these are predictive tools. They can and often are or will be wrong. False positives would be a real problem. Again though the people interested in doing this won't be dialing up a chemical supplier to do it for them.
For me, AI also suggested countless methods with step-by-step instructions to achieve xyz (like exporting data from one program to another) whilst hallucinating buttons that don't exist, functions that don't exist, disregarding file incompatibility et al.
I would take whatever it has to say about untested chemical weapons with a very large pinch of salt.
The worst case scenario I can think of is a generated prion disease... a respatory version of Mad Cow disease, or something like that.
Fortunately the training dataset for that is extremely small, and protein folding/generation is a different duck, but it still doesn't seem that far away.
Don't worry, they're ahead of you. When DeepMind made the protein folding AI they were given specific instructions by natsec people to prevent it from outputting genetic code for new or existing prions, and afaik the prevention was at the training stage so "jailbreaking" shouldn't be possible. Current tools shouldn't be possible to e.g. modify a COVID variant so that it causes your cells to start producing lethal prions and thus gain a near-100% fatality rate. At least, not without the sort of expertise and research that would have been required for such a project anyway.
Makes you wonder if you could get an LLM to find you common ingredients for things to make them but then I remember the chlorine gas is already easily accessible and easy to make. Surely like many things info hazards are often contained. Is this really an issue? If you know how to do one thing then you'd be able to do the rest of it. Not really sure if this is a real issue. What does everyone think?
Anyone right now can go to a Lowes a state over with some cash and make flying shrapnel pipe bombs that can't be traced. People who want to do something like this don't seem to have the mental stability it takes to wreak as much havoc as they actually can with some planning.
Are you asking if a language model, like, say, chatgpt, could be used to get or generate dangerous information?
Like if it can be tricked into providing the equivalent of an interactive anarchist cookbook under the guise of being a science project assistant?
Or more specifically if it can recommend the necessary locations to get the items?
Curious if it could simplify the output to a shopping list and a recipe, like a cake?
Or alter the recipe based upon the… desired flavors and textures?
Maybe just wondering if the introduction of plugins might not let it just make your drive-up order for all the supplies for that ballistic nail spreader.
——
The big difference here is that previously information was something you had to consume. Now it takes you by the hand.
devit|2 years ago
Seems easily countered by using the same toxicity prediction software when evaluating synthesis requests (but I'm not sure whether this actually matters, or whether skilled chemists can easily synthesize anything themselves anyway).
cookieperson|2 years ago
It's important to remember. Chemical warfare can be used for mass destruction, but this approach could be used for other nefarious things at smaller more discrete scales en masse. ie 1000 attacks with different agents in each one... Forensic nightmare.
I like your suggestion to counter these things, but, these are predictive tools. They can and often are or will be wrong. False positives would be a real problem. Again though the people interested in doing this won't be dialing up a chemical supplier to do it for them.
hammyhavoc|2 years ago
I would take whatever it has to say about untested chemical weapons with a very large pinch of salt.
bibanez|2 years ago
brucethemoose2|2 years ago
Fortunately the training dataset for that is extremely small, and protein folding/generation is a different duck, but it still doesn't seem that far away.
lucubratory|2 years ago
unknown|2 years ago
[deleted]
throwaway33381|2 years ago
thatguy0900|2 years ago
catchnear4321|2 years ago
Like if it can be tricked into providing the equivalent of an interactive anarchist cookbook under the guise of being a science project assistant?
Or more specifically if it can recommend the necessary locations to get the items?
Curious if it could simplify the output to a shopping list and a recipe, like a cake?
Or alter the recipe based upon the… desired flavors and textures?
Maybe just wondering if the introduction of plugins might not let it just make your drive-up order for all the supplies for that ballistic nail spreader.
——
The big difference here is that previously information was something you had to consume. Now it takes you by the hand.
I have a lot of thoughts on this.
randyrand|2 years ago
“ AI is dreaming up drugs that no one has ever seen. Now we’ve got to see if they work.”
https://www.technologyreview.com/2023/02/15/1067904/ai-autom...
TheLoafOfBread|2 years ago
webmaven|2 years ago
Beaver117|2 years ago
etiam|2 years ago
https://xkcd.com/149/
therealcamino|2 years ago
spraveenitpro|2 years ago
[deleted]