top | item 5506623

I want to write software that helps kill people.

40 points| edsu | 13 years ago |gist.github.com | reply

from the Philosophy in a Time of Software discussion list: https://groups.google.com/forum/#!topic/philosophy-in-a-time-of-software/SnV8qMUQc3Y

57 comments

order
[+] anigbrowl|13 years ago|reply
That you have a computer at all is in large part due to the overwhelming demand for accurate ballistics data and codebreaking during WW2. It's very easy to play '6 (military) degrees of separation' and conclude that virtually any human activity supports the killing of people. Like baking at home? Your bakery hobby slightly reduces the demand for manufactured bread, thus making it more affordable to a heartless military machine. And so on.

If you're not actively and directly involved in military activities, there's little that you can do about the knock-on effects of whatever it is that you're engaged in; and even if you can quantify the human cost of your activities, it needs to be offset against the positive externalities. For example, the same software that allows Palantir to identify 'bad guys' who are potential targets for military attack is equally capable of identifying peacemakers or other constructive individuals who are good candidates for receiving financial aid or suchlike.

Technology itself is neutral. Don't use it as a stick to beat yourself over the imperfect state of the world.

[+] wildermuthn|13 years ago|reply
Technology is not neutral. Technologies are designed for discrete purposes. A hammer is designed for, among other things, hammering nails, not eating. And as it has been said, "When all you have is a hammer, all your problems look like nails." A CIA drone is not a neutral technology.

Kevin Kelly (a former editor of Wired) wrote a book called "What Technology Wants." He reminds us that technology does have a purpose, does influence human behavior, and may in fact be changing human nature.

Kudos to the OP for taking a reflective, responsible, and principled stance. We should all be so discriminating.

[+] fennecfoxen|13 years ago|reply
The question that this comes down to is a question of one's world-view and philosophy of responsibility, guilt, and concept of sin. It says something about how you conceive of those things if you trace responsibility from "meaningful contribution to a NoSQL storage engine" to "code was executed by a system used by people to seek terrorists and justify an eventual drone strike".

Due to relativism, I can't tell you whether your view is intrinsically right or wrong in a manner that all people are guaranteed to respect, but I can tell you: I would be upset if you used that logic to find fault with someone else who wrote NoSQL database engines, and blamed them for killing people (terrorists or no). I think it would be horribly unfair and modestly ridiculous - and if you agree with that when it applies to other people, consider whether you are, in fact, giving yourself a fair shake here.

[+] rayiner|13 years ago|reply
The fact of the matter is that war is a basic function of human societies, as much as food production or waste disposal. Unsurprisingly, it thus taps the ingenuities of the society in the same way as those other areas. It both benefits from ingenuities intended for other purposes, and results in inventions itself that can be used for other purposes. And for better or worse, war is a great way to get the public to support R&D spending. DARPA could not be what it is without being a part of the DOD. See: http://tech.mit.edu/V114/N29/dod.29n.html, http://web.mit.edu/annualreports/pres07/12.00.pdf, http://en.wikipedia.org/wiki/DARPA_Grand_Challenge.

And let's face it, its something you benefit from tremendously. You do not want a society in which the US does not have the power to kill everyone. That is a society in which your lifestyle is not as nice as it is now. Indeed, with the challenges we face in the coming century (oil, water scarcity, etc) as well as the ascension of India and China as world powers, our ability to kill people is going to be more important than ever.

[+] smtddr|13 years ago|reply
> The fact of the matter is that war is a basic function of human societies, as much as food production or waste disposal.

That makes me sad. Does anyone on HN(besides me) believe that one day humans will be able to rid themselves of war? That one day we'd find a solution to these issues that don't involve sending a group of armed people to kill?

[+] venomsnake|13 years ago|reply
"It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter."

Nathaniel Borenstein

That sums it up for me.

[+] vubuntu|13 years ago|reply
You also give money to kill people (You do pay taxes, right?). How come you are not bothered about that also?

It's pointless to think about the impact of a micro action on a macro event. Especially macro things such as war. There are so many different factors that culminate in a war.

And about the good/evilness of war, that has a been a debate since ages. One can never win this argument.

[+] wildermuthn|13 years ago|reply
Plenty of people refuse to pay taxes for precisely this reason, or deliberately seek lower salaries that do not pay federal taxes.

They believe, correctly, that macro-events are the culmination of micro-events, and that change begins on the personal level. It takes just a handful of dedicated people to start revolutions. Startups should know that.

[+] brown9-2|13 years ago|reply
This logic seems like quite a stretch. I'd imagine the person who designed a hammer or a screwdriver isn't preoccupied with the fact that it might be used in helping to build a bomb factory.
[+] binarymax|13 years ago|reply
I wrote, but never published [1], a variation on the MIT license that has specific clauses stating that the software can only be used by parties who dont own/manufacture/trade firearms or ballistics. I never published it because I am not a lawyer, and it probably would have come across as an elaborate troll. If anyone is interested, I'll release it.

[1] https://github.com/binarymax/Non-violent-License

[+] gyardley|13 years ago|reply
It's your license, but owning firearms somehow makes you violent? Certainly sounds like an elaborate troll to me.

Even if you could somehow iron out the kinks, the end result of your sort of action is a host of additional licenses that bar the various bête noires of their various authors, eventually leading to code that simply can't be used together because of conflicting moral obligations. I really don't want to have to run a program just to sort out the ethical dependencies of the stuff in my Gemfile.

[+] bane|13 years ago|reply
You might be interested to note that the world's largest R&D firm [1] has a charter where they can't research offensive weaponry, but can research "dual-use" technology. e.g. if you can develop techniques to turn a cancer drug into an aerosol for oral delivery, you can use the same technique to turn a mildly infectious contagion into a bio-weapon.

1 - http://www.battelle.org/

[+] moron4hire|13 years ago|reply
What about hunters and police officers and security guards and sport shooters?
[+] guelo|13 years ago|reply
This has nothing to do with open source. It's just a consequence of living in a warmongering superpower. Considering that a good percentage of America's GDP goes towards killing brown people overseas, just about every economic action you take as an American is contributing towards killing people. If you extend certain moral frameworks far enough, just living in America is an evil act.
[+] venomsnake|13 years ago|reply
Come on ... be real ... america spends roughly 600 B USD on military and related stuff. And they kill what 200-300K of civilians and combatants overseas per year top.

So this means that a brown person's life costs the government what 3 million USD per person to take. That is grossly inefficient. I am sure that if they just offer the dictators and warlords the money directly to kill their own populace they will manage to bring down the bill to 10K per person.

The goal of the US military is not to kill people (it is just not good enough) or defend the country (there is no real threat and terrorism is a joke) - it is to waste GDP in the best Orwellian way possible in which it excels.

[+] mschuster91|13 years ago|reply
I vaguely recall license terms which disallow anyone from using the code for military purposes - maybe it's time to adopt stuff like GPL/CC by extending them with an (optional) "do not use this software in machines/systems which have the power to kill/hurt people, or enable people to do so"
[+] rmc|13 years ago|reply
Terrible idea. It would also be non backwards compatible with all existing open source / free software licences. It would be a legal minefield and stop a lot open source software.
[+] wereHamster|13 years ago|reply
Like the JSON license. MIT with this additional clause: The Software shall be used for Good, not Evil.
[+] kyrias|13 years ago|reply
Adding it to the GPL would be pretty ironic.
[+] VLM|13 years ago|reply
You're confusing military purposes with "have the power to kill/hurt people".

Any desktop fits #2 if I open it up, open up the power supply, and lick the output of the mains rectifier in the switching power supply. Or consider a laptop/tablet having an uncontrolled thermal battery reaction (aka bursts into flames) on an airplane over the middle of an ocean.

Somewhat more directly, my dentist can't use a "sorta-open-source" calendaring app because people have died in dental offices from allergic reactions to injected dental antibiotics/anesthetics. Its kinda rare, but has been recorded to happen.

Simply naming or defining organizations is not as simple as it appears. Your freedom fighter group is someone else's terrorist is someone elses pacifist organization.

You need a cop-out for intention and rarity. By the time you add enough cop-outs that anyone can actually use it, you've almost certain cop-out enough that the .mil can also use it.

[+] wildermuthn|13 years ago|reply
Check out Stanford's Encylopedia of Philosophy entry on double-effect: http://plato.stanford.edu/entries/double-effect/.

Essentially, double-effect means that your good actions can have unintended yet foreseen evil consequences. If you make a hunting rifle, and someone uses that to shoot children at a school, are you responsible? Not an easy question to answer.

[+] ctdonath|13 years ago|reply
Very easy question to answer. The answer is "no". There was no ill intent, no expectation of misuse, no "mens rea" ("guilty mind"). The intended and expected market is very large and overwhelmingly upstanding. The good done far outweighs the freakish non-sequitur bad. You don't not do something just because, with a twisted line of reasoning, someone can conceive of 1 in orders of magnitude of orders of magnitude chance it might be abused.

Someone driving a car into a crowd does not make it Ford's fault. Society does not ban cars just because 50,000 people a year are killed by them.

[+] rmc|13 years ago|reply
Guns are an interesting case. Thee are many tools (eg computers, knives, hammers) that can be used to kill someone, but most of the uses of them are not killing people (more tomatoes are cut with knives than people killed). I'm not so sure about guns. Are most of the instances of guns being fired being used to kill/hurt someone (or practice to kill/hurt someone)?

Perhaps it's the European in me that can't wrap my head around the American idea of guns. To me "Guns are dangerous" seems obvious.

It's an interesting cultural difference.

[+] tshepang|13 years ago|reply
I suspect that future generations will be disgusted by the fact that we find it okay to shoot conscious beings for amusement. Am generalising of course.
[+] pessimism|13 years ago|reply
I was once at a guest lecture by Bjarne Stroustrup (creator of C++ for those who don’t know), and during Q&A, someone in the audience asked him how he felt about the fact that his work was used to kill people.

(The rationale behind the question was based on fighter jets, to some extent, using software written in C++.)

The question was not tongue in cheek. Everyone in the auditorium laughed, though.

[+] ceautery|13 years ago|reply
I want to hack your software such that it always returns null. And then write software that identifies better distribution methods of food and water to famine and drought-stricken areas. Or maybe find the best areas to reseed of those that have been deforested.
[+] jimfl|13 years ago|reply
If you work at McDonalds, you probably fed a soldier or a weapons designer, or CIA field operative. You'd have to drop out of productive society entirely to escape this dynamic.
[+] moron4hire|13 years ago|reply
This is the nature of living in a free society (or at least what parts of one that are left). You don't get to dictate what other people do. If you release your code "free to the world", you have to open your idealistic eyes and realize that that might include things you aren't comfortable with. And to maintain consistency with the views that led you to opening your code, you have to be okay with it. That is the unfortunate conceit of most people who claim to be tolerant and open minded, they often mean "but only for things I approve of".
[+] peterwwillis|13 years ago|reply
It's even worse than that! I hear people use hammers to build buildings where people work on military projects that eventually kill people! Clearly, we need to stop people from using hammers for evil. Boycott the hammer industry today.
[+] Dewie|13 years ago|reply
> I'm reasonably certain I would be orders of magnitude less well off and less happy if open source didn't exist as a concept.

Right... what happened to just saying 'hundreds/thousands of times better off#' in hyperbolic sentences? Too pedestrian?

#Of course you can replace 'better' with any fitting adjective.