top | item 27528243

Lethal Autonomous Weapons Exist; They Must Be Banned

152 points| doppioandante | 4 years ago |spectrum.ieee.org

180 comments

order
[+] marcus_holmes|4 years ago|reply
This feels like the same thing that happened with tanks. The end of WW1 saw tanks deployed effectively. WW2 was then fought using entirely new tactics and strategies enabled by tanks (not least "blitzkrieg").

Britain and France, notably, hadn't really worked out how to use them effectively and so were left fighting the last war, horribly disadvantaged until they worked out how to catch up (though air power being developed at the same time also had an influence).

So now we have drones, and all those incredibly expensive fighter planes are useless. Aircraft carriers will be replaced with cheap drone swarm carriers, so naval strategy will need to change. Anti-tank drone swarms will make conventional armour more or less ineffective. And so on.

The good news is that there will be anti-drone swarms too. And so we'll end up fighting battles mostly by destroying drones instead of people. Once you've knocked out all the enemy drones, they'll surrender because humans can't fight drones.

Interesting times.

[+] epicureanideal|4 years ago|reply
> Once you've knocked out all the enemy drones, they'll surrender because humans can't fight drones.

Or will lead to blending in with civilians, more advanced camouflage, or “hugging the enemy” tactics (staying close enough that they risk harming their own people). Or more terrorism rather than open warfare.

It might be that with these killer drones, we will look back on WW2 style armies in the field as equivalent to 18th century musketeers lining up and firing volleys at each other in bright colored shirts.

[+] stickfigure|4 years ago|reply
> those incredibly expensive fighter planes are useless. Aircraft carriers will be replaced

I'm not fully convinced. The advantage of drones is that they are relatively cheap. That automatically comes with significant range and speed limitations; after all, as someone else mentioned, we already have long-range semi-autonomous drones called missiles. They aren't cheap.

Naval engagements, not to mention aircraft engagements, happen at distances that are just not in the reach of cheap drones. Someone might get a surprise shot at an aircraft carrier in peacetime, but probably not while alert on the open sea. And I'm sure ships will be upgraded with defenses (if they haven't already).

Tanks and infantry, that's a more interesting problem.

[+] falcolas|4 years ago|reply
Nah. Humans can't fight cruise missiles either, yet it hasn't changed to where one side gives up when the other has cruise missiles.

Humans aren't that logical.

[+] petermcneeley|4 years ago|reply
This analysis is slightly off because WWI and WWII were great powers fighting each other. If the great powers were ever to directly fight this would still be nuclear war. Nothing about "swarming bots" has significantly changed this domain.

Where bots and swarm bots would be used is likely in asymmetric warfare.

"Aircraft carriers will be replaced with cheap drone swarm carriers" I would argue that Aircraft carriers are the size and cost due to large munitions and maintenance constraints.

[+] GuB-42|4 years ago|reply
> So now we have drones, and all those incredibly expensive fighter planes are useless.

Not useless. The idea seem to be that fighter planes will become something like a command center for the drone swarm. There is value in having a human on site, to take quick decisions for instance, and links to drones can be jammed.

And if you look at the F-35 program, you'll see hints that it is the way we are going. It is a stealth fighter with a heavy emphasis on communication systems. Perfect for managing a fleet of drones. Ok, the execution is far from perfect, but that's another debate.

In the end, it is not entirely unlike the current situation. The difference is that these drones blow themselves up on arrival and are called missiles.

Oh, and swarms of drones won't be cheap, that's the army we are talking about. "Predator" style drones are multi-million dollar machines. A small missile like the Hellfire costs around $100k apiece. Forget your idea of a swarm of hobby quadcopters.

[+] spywaregorilla|4 years ago|reply
> The good news is that there will be anti-drone swarms too. And so we'll end up fighting battles mostly by destroying drones instead of people. Once you've knocked out all the enemy drones, they'll surrender because humans can't fight drones.

They can still fight the guerilla war among citizens. That's been an incredibly effective strategy because it's so expensive to fight. One might wonder if drones are the key to fighting these wars cheaply. If all you need is a small local base and then a thousand drones to surveil a populace... Well that sounds pretty nightmarish for the citizens, but it would certainly change the nature of war.

[+] nickthemagicman|4 years ago|reply
> And so we'll end up fighting battles mostly by destroying drones instead of people.

This is one piece of possibly good news that it seems like everyone is overlooking.

Automation/Remote control applies not just to drones, but tanks, boats, other aerial vehicles, and maybe troops(after seeing that Boston Dynamics dog robot) etc. Seems like it will be a small squad of highly advanced human troops to move in after the Remotes to secure and maintain control of the area.

So many lives will be spared during wars.

However, this may be unbeatable tech by humans, so any future revolutions will be easily quelled by a gov't with a few drone pilots piloting a drone swarm in a room a thousand miles away.

[+] caseysoftware|4 years ago|reply
This is a rough explanation of the "generations of warfare" where our tactics are defined by the tools and weapons we have available. Unfortunately, we (humans, not just the US) tend to then treat those as the "right" tactics even as the underlying tools and weapons change so we're always fighting the last war.

Cheap, autonomous, easily-deployed tools - weapons drones, sensors, etc - are a huge change that some are adapting to leverage but few have adapted to respond to.

Which is scary considering the "generations of warfare" model where in fourth generation warfare, the boundary between civilian and combatant is blurred..

[+] grey-area|4 years ago|reply
> And so we'll end up fighting battles mostly by destroying drones instead of people. Once you've knocked out all the enemy drones, they'll surrender because humans can't fight drones.

A war like no other?

[+] ReptileMan|4 years ago|reply
And I will spend the weekend developing autoaiming gun battery that shoots the .22LR that needs to ground one of those birds and we are at stage 1.
[+] jayd16|4 years ago|reply
I kind of doubt drone vs drone will be all that effective. Have anti-missile systems ever been all that effective?
[+] sabhiram|4 years ago|reply
Directional EMP development, here we go!
[+] hermitcrab|4 years ago|reply
>Once you've knocked out all the enemy drones, they'll surrender because humans can't fight drones.

And what happens if these autonomous drones aren't programmed to accept surrounder (as seems likely)?

[+] onion2k|4 years ago|reply
Once you've knocked out all the enemy drones...

Then the enemy will build, or buy, more drones. Well-funded armies or terrorist groups don't lose because they run out of weapons.

[+] koonsolo|4 years ago|reply
I see you have absolutely no idea what you are talking about.

A drone cannot effectively damage a tank or any other major equipment.

The payload a swarm-drone is just way too small. Why have 100 needles when you can have 1 sword.

We have rockets, which are way more effective than any number drones.

A "drone" that can do proper damage is already the size of a plane. Anything less is only useful for recon.

[+] thesuperbigfrog|4 years ago|reply
The problem with arms races is that they are races: no one wants to be left behind.

What is likely to happen is any major nation-state that signs a treaty agreeing to a ban will STILL secretly be researching and building autonomous (or nearly autonomous) weapon systems because they do not want to be left behind.

[+] omginternets|4 years ago|reply
I would add that it's also very difficult to study and develop countermeasures without cultivating cutting-edge knowledge of how such weapons are designed, built and operated. Doing that means you eventually need field data. The slope is extremely slippery, but it's also dangerous not to traverse it.
[+] marvin|4 years ago|reply
It would be a disaster to enter a military conflict with an adversary that has autonomous weapons if you don’t have at least effective countermeasures yourself.

I suppose nuclear states could in principle rest on their laurels, but at least from a layman’s perspective it doesn’t give an impression of being strategically sound.

But obviously it’s a scary rabbit hole to enter. It’s not fun when game-theoretical considerations or human nature seems to make dubious technological developments inevitable.

[+] marcinzm|4 years ago|reply
They also need to research them to learn how to counter them if an enemy does use them. Given that most of the conflicts of the last few decades were not with states but with groups the chance of someone using them is high. Especially as technology becomes more accessible.
[+] xvilka|4 years ago|reply
Winner will be the one who will design biological equivalents - genetically modified eagles or something similar.
[+] CoolGuySteve|4 years ago|reply
One of the reasons the attempted drone assassination of Maduro in 2018 failed was because police were using radio jammers: https://en.wikipedia.org/wiki/2018_Caracas_drone_attack

It's fairly easy and cheap to make an explosive drone and train a neural network to drive it using off the shelf components and open source software. It follows that it's inevitable that at least semi-autonomous drones will be made to counter radio jamming.

So while the best outcome a treaty could hope for is to prevent mass manufacturing these things by some countries, any motivated small country/individual will be able to craft them.

[+] gonzo41|4 years ago|reply
Drones can't beat nylon nets or other low tech countermeasures. I think anti drone water cannons will outpace drone assassinations. And more than both, regular bombs and guns will still do most of the killing.
[+] todd8|4 years ago|reply
Banning autonomous weapons is a good idea, but history indicates that the bad guys might very well continue to develop them while claiming that they are not. For example, see the biological weapons program pursued by the former Soviet Union in secret despite being signatories of the international ban on biological weapons[1].

[1] https://en.wikipedia.org/wiki/Soviet_biological_weapons_prog...

[+] some_random|4 years ago|reply
There's an interesting similarity between autonomous weapon systems and ransomware discussions. Both have just come to the attention of the mainstream audience (or audiences outside their usual niche) and discussion in those spaces is about where it was 5 years ago.

Also I'd really expect HN of all places to understand why banning what is basically computer vision based targeting systems is absurd.

[+] ethbr0|4 years ago|reply
Technologists tend to underappreciated relative utility as a driving popularity metric -- does it accomplish a job I need, better / cheaper / faster?

My thought is that ransomware is seeing a resurgence because corporations are increasingly internetworked and hybridized with the global internet, because they have to be. So ransomware has begun to outweigh espionage as a method for extracting money from corporations.

Drones (as used so far, in the remote suicide version) solve a key problem many states have: lack of effective tactical ISR. The US and major powers do not have this problem.

Autonomous drones solve... what problem?

My inclination is that they'll deliver the same value as current-gen ML systems: mass, automated intelligence collection & basic analysis. As well as continuous monitoring and triggering on a pre-configured event (e.g. truck leaves this building, launch missile).

Killing something, once you know what and where it is, is not a problem most militaries have.

Training and paying and allocating large numbers of soldiers to do basic intelligence trawling is absolutely a problem most militaries have.

I think you see this in the US drone approach evolution. Shifting from a single vehicle approach to a survivable system with attritable assets communicating back through stealth communication hubs. Because persistence is the real value.

[+] x86_64Ubuntu|4 years ago|reply
>...Also I'd really expect HN of all places to understand why banning what is basically computer vision based targeting systems is absurd.

On the contrary, I would expect a site with a usersbase that screams incessantly about "privacy" and has an intense hatred of data-mining dark patterns, dislikes opaque ML models, would be against the lack of privacy and data-mining culminating in an execution by the state.

[+] tclancy|4 years ago|reply
I would hope (but definitely not expect) HN commenters to see past this and realize the problem isn't computer vision or firepower. It's tying those two together, writing some code and cutting it loose on humanity. Put it this way: if you wrote the code for an autonomous machine gun platform, would you park it in your house? I'm twenty years into this and I wouldn't trust my code that well.
[+] stfp|4 years ago|reply
Just like with chemical weapons, banning them makes total sense to avoid overwhelming mass production and ubiquitous use.

You'll still see them used in targeted assassinations and some countries will have illegal military stockpiles, but that's very different from these things being treated like regular guns/ammo, being commercially available or legal to build.

[+] michaelt|4 years ago|reply
Are these drone-based 'lethal autonomous weapons' markedly different to cruise missiles?

Or is the alarm here about the fact that every man and his dog will be able to get their hands on them, when they cost $1000 instead of $1000000?

[+] neurobashing|4 years ago|reply
I guess the answer to me is "it depends". Modern cruise missiles are pretty much drones, but at the same time they're always what I guess I'd call "weapons of state". Big, expensive, and part of a large decision-making loop. You can't just lob one; someone signed for it. Attacks are planned in advance at high levels.

whereas this "slaughterbot" idea is smaller, low-cost, and with far less top-down control.

At least, as I understand it.

[+] nick238|4 years ago|reply
I think the latter. For what it's worth, the 'arms' that are still protected by the 2nd Amendment are still approximately the same as in 1787 (line of sight, bang bang), as you're legally restricted from owning guided bombs, cruise missiles, nuclear weapons, etc, but even aside from that, you're practically restricted because of limited availability and/or extreme cost.

On the flip side, the hardware for Slaughterbots is basically here: TinyWhoops cost ~$100, say ~$100 to upgrade the ARM processor to one that can process realtime video, ~$100 for a shaped charge. So the hardware is super cheap, and once developed, the software to tie it all together has nearly no incremental cost. Once it's out there, that's the genie leaving the bottle.

[+] zokier|4 years ago|reply
That is my question too. Tomahawk missiles have been fielded since early 80s, and are very much fire and forget autonomous weapons that can track targets based on sensor data (i.e. not reliant on GPS or other external guidance).

Maybe the small difference is that cruise missiles generally are not anti-personnel weapons; maybe the article is seeking a ban in the same sense that Ottawa treaty banned anti-personnel mines but allows anti-tank mines. But I think this is something that would need to be explicitly addressed instead of just spouting sky is falling rhetoric.

[+] Invictus0|4 years ago|reply
Weapons that have been successfully banned in the past typically had an alternative technology that it could be substituted with. I'm not sure you can say the same of lethal drones. In the case of Azerbaijan vs Armenia, the sum total of the fighter aircraft of BOTH nations combined is just 5. For poorer nations, drones represent a democratization of the battlefield; an extraordinarily cheap technology with a lot of firepower per dollar. I don't think these countries will be willing to give this up easily.
[+] credit_guy|4 years ago|reply
How silly. How can such a ban be enforced? Will Al Qaeda and ISIS sign the treaty? Will Assad sign it? Will he respect it any more than he respected the ban on chemical weapons?
[+] superbaconman|4 years ago|reply
Is this the official position of ieee? It's seems both unrealistic and unconstitutional (at least within the US).
[+] 5560675260|4 years ago|reply
I, for one, welcome our murderbot overlords. If we can't stop using drones to kill people, we should at least try to reduce collateral damage. With some advancements autonomous drones could be more precise and reliable than human operated ones, meaning that as an outcome fewer weddings will get blown up.
[+] mam3|4 years ago|reply
This is dumb and childish on so many levels.

You will NEVER ever prevent bad guys from creating them by "banning" them.

Look at drugs: all this war / bans / illegality, and the cocaine traffic never ever went down in decades (maybe during covid).

Best course of action is to invest in them to not be left behind

[+] Wavelets|4 years ago|reply
I agree 100%. So naive. Just think of a world where we (US, EU, etc.) stop investing in this technology while Russia, China, etc. continue.

It really burns me to hear AI professors rail against “autonomous weapons” yet go work at Facebook, Google, etc. Information, especially manipulation of it, is a weapon too.

[+] gpm|4 years ago|reply
Autonomous weapons should be embraced, otherwise less scrupulous people will embrace them, and they will win. Non-autonomous weapons will simply not be able to compete long term.

Actually we wouldn't even be able to compete today, things like missiles are already autonomous, and if they weren't we would simply be unable to compete in any battlefield where radio was jammed, and it would give a large force multiplier to adversaries even in battle fields where radio is not jammed.

[+] ru552|4 years ago|reply
[+] mdorazio|4 years ago|reply
These countermeasures generally only work against drones that are not fully autonomous since they rely on disrupting communication with a ground controller. If your drone is perfectly happy to loiter and kill people based on facial recognition and some basic parameters, dronegun won't work. The other point to keep in mind is that cheap lethal drones are often designed to operate in swarm mode rather than individually. It's pretty difficult to knock out an autonomous swarm with weapons designed to target one drone at a time.
[+] ccsnags|4 years ago|reply
In the book “The Diamond Age” by Neal Stephenson there are mist-like swarms of nanobots that act as an immune system for regions against malicious nanobots.

I can picture there being something like this with drones.

There are problems to this. The major one being that, as mentioned by the posts on this thread, you have to develop the weapon to really know how to counter it.

[+] Digit-Al|4 years ago|reply
I see lots of people offering elaborate solutions to take down a swarm of drones. I may be wildly off the mark here but would it not be feasible to fly over them with a jet fighter and let the supersonic shockwave take out a lot of them?

I am thinking of some other more low tech solutions to whittle them down as well: firing a weighted net; sticky silly string type stuff to tangle and gum up the rotors.

[+] CountDrewku|4 years ago|reply
They seem to be only presenting the negative aspects of "slaughterbots".

Could they not potentially be better than human controlled devices since they should be able to more accurately target things and create less collateral damage?

Of course, there's always the doomsday terminator scenario where they go haywire and decide to start targeting all humans....

[+] nabla9|4 years ago|reply
I don't fully understand how there is real difference?

If you give a swarm of drones a kill zone and time limit, then they try them to kill every human in the area, how is that so different from MLRS salvo?

Cruise missiles are already slaughterbots. You just give them location and they kill everyone in that location.

[+] alexf95|4 years ago|reply
It always has been scary to me to how weapons evolve over time. Even though they are (thank god) mostly not needed they still keep developing further to be more lethal and efficient just to be "prepared" to what might happen in the future.