I think the imposition of this burden on engineers is the result of our belief that our legislators are too incompetent or unwilling to act in the fact of clear moral hazards.
Tech has a very strong history of conscientious objectors. They're not the problem. Other institutions need to do their part.
I've long thought that tech needs like.. a guild or trade union, or a professional association of some kind. The problem though, is that it's so easy to get into, so there's no easy way to "police" ourselves, because tech has been democratized so well-- Anyone can do a code boot-camp and be up and running within weeks. Not that that's a bad thing, it just makes it harder to hold developers to a code of ethics or anything like that...
Maybe we just, as a society, need to adopt morality and ethics more deeply into our cultural DNA.
Canada has a professional software engineering organization, at least in some provinces. That said, they talk a lot about ethics but then award first place at a local engineering competition to a team that tracks people's position and movements without them knowing using their cell signals (for use in airports), so maybe it doesn't do any good. Great technical work by them mind you.
While it's true that getting "up and running" with a simple project is only a matter of getting a working laptop and internet connection (and a bootcamp if one is available to you), getting into the position where you could build (or not build) a system capable of broadly violating human rights on behalf of an oppressive government is not an opportunity most workers have. If you're interested in the sort of union that could help you take action in your workplace I'd recommend looking into the Tech Workers Coalition -- there's an active community online and probably a branch near you.
Further down in this thread the discussion has broadened to worker rights in tech-adjacent jobs (eg. amazon warehouses). It would be great if we could have one big union that fights for the rights of all workers, and if you think so too you should look into joining the IWW.
Yes, our society does need to adopt morality and ethics as guiding principles in our work, and organizing your workplace is not some orthogonal venture but is actually part of that goal.
Why put this on programmers? The guy making the decision to make Dragonfly is a Stanford MBA with a materials background. It's not some working class kid who made it through a Javascript bootcamp that is making decisions about the direction of future Google products. So, why don't we let the MBAs be the ones to form an ethical organization to police themselves.
Just because there's a professional organization that pushes a set of ethics, does not mean they will be anything resembling the type of ethics you are envisioning. Building tools, your ethics generally get framed with regards to your client - not the subjects your tools will be used on.
Traditional engineerings have longstanding concepts of ethics, plenty of professional organizations, and even professional licensure. This does not stop mechanical engineers from adding to USG's weapons disparity or civil engineers from constructing more profit-center cages.
I'm a software engineer largely because my father and grandfather were union laborers. My and my cousins' generation was the first in our family to be able to go to college. And it was precisely because the union was so easy to get in to that my father and grandfather were able to get good paying jobs with benefits without a college education. A low barrier to entry to a profession is no reason not to build a union.
I think the challenge is a bit misunderstood online. The challenging part by far with forming something like a trade union is to gain the influence, not how to wield it. It is having a high participation rate, a strong organisation, institutions and a war chest is the hard part. Actual influence will then be largely automatic just by existing. In general only weak organizations need "activist activities". A strong union's activities would be ongoing negotiations, legal cases, education, conferences, media production etc. That would form the culture more than any sort of code.
The UK has the British Computer Society -- though I'm not actually sure what purpose they serve.
I was a student member for a couple of years, but beyond the occasional Government IT job advert which required BCS qualification and accreditation, they seem to be extremely quiet (or not very good at advertising).
> The problem though, is that it's so easy to get into, so there's no easy way to "police" ourselves, because tech has been democratized so well-- Anyone can do a code boot-camp and be up and running within weeks. Not that that's a bad thing, it just makes it harder to hold developers to a code of ethics or anything like that...
Moreover, "coding" is a cross-cutting concern in that a lot of people who aren't formally programmers have been sold on the idea of being able to automate parts of their job by writing code, and I'm not sure how you'd yoke all of those disparate fields (especially academics outside of industry) into a single labor organization.
Why does the fact that people can join easily mean there's less interest in a union? Should Amazon factory workers not form a union if they find it's easy to join the ranks?
"We are a small - as-of-yet unincorporated - nonprofit providing pro bono consulting on algorithmic and policy issues arising from the proliferation of: Statistical inference / Automated decision making - often called 'AI', ..."
From the author's website, listed at the bottom.
Imagine if journalists and other people posting to the www called "AI" what it actually is, instead of constantly portraying it as something futuristic to capture people's imaginations.
Even terms like "statistical inference" and "automated decision-making" could probably be explained using more common language to be comprehensible to the general public.
Compensation, including perks, is usually very good in tech in comparison to many other industries. Most people rationalize and convince themselves that they're not really doing bad things, and that it's someone else who's the problem.
In some companies, the culture is also that of being oblivious (or acting oblivious) to the harm caused by the work. Facebook is the best example of this. Has there been any conscientious objector in that company? People like the WhatsApp and Instagram founders, who grew a conscience after getting billions in their pockets, don't count (even if one or two left some money on the table when quitting).
Money tells many a true tale of what goes in in people's minds.
The article, sadly, does not mention Facebook and all the surveillance that it enables and grows, along with oppressing people by forcing them to use "real names".
While I agree that individuals should consider the ethics of the projects that they work on, I think it's unfair to place the entire burden on them. Those in management who make the product requirements in order to sell the software are in the best position to evaluate the ethics of how software will be used. Why haven't I read a story about management conscientiously objecting to something?
I think this is s bit naive. Americans and EU citizens could conceivably follow some protocol such that what they produce will not go against certain principles (if we even presume no such thing as dual use tech), but that would do little to stifle and control developers outside this sphere. Vladimir, Jinping, Jeong-Un, Narendra, etc., aren’t about to handicap themselves and have some realization, oh, you know what guys, we should be respecting human rights, we gotta stop.
I get the point you're making, and it's valid... but "Vladimir, Jinping, Jeong-Un, Narendra, etc" is a bit offensive. One's ethnic background, and even citizenship, does not define one's morality. Take a look at the list of names on http://neveragain.tech/ - it's not all Americans and Europeans.
A conscientious objector is an "individual who has claimed the right to refuse to perform military service" [0]
I believe that it is dangerous to conflate job choice with conscientious objection.
Everyone should be held accountable for what they spend their day contributing to - this shouldn't be a special case.
Comparing this to a situation in which someone doesn't want to participate in compulsory murder, and in some cases risks being killed themselves, is not helpful.
On the one hand, I agree with you that it isn't helpful.
But on the other hand, there is a plausible parallel. The reason for strong controls on privacy is because every couple of generations governments tend to do a lot of damage to their own citizens.
If I thought the worst that was going to happen was the risk of being wrongly accused, public shaming or something, then privacy isn't really worth it. I trust law enforcement to be generally correct in assessing the situation, and if you are doing something shameful you may as well be shamed.
Privacy is important because it is one layer of protection against institutions participating in the murder of citizens, seen in Germany, China, Russia in the last generation and China + others this generation. Who knows for next generation? Change is very quick - there is no law of nature that says it won't be an Anglophone country. We can get thingy about whether a conscientious objector has to be the last person pulling the trigger or not, but in practice that distinction is not an important one.
Worth noting that probably everyone here has seen something like this happen. Not sure if it is specific to tech, but some of the egos and pathologies are extreme.
This article is premised on the idea that basically everything the Chinese government does is unethical, which is debatable. For one, the Chinese government is largely supported by its population. There might actually be engineers working on Dragonfly who do not believe the project is unethical. It's not that black and white...
How would one know if the Chinese government was supported by the population or not, given that anyone expressing a contrary opinion is likely to be a thorn in the eye that is China's surveillance state?
Being largely supported by its population is not a valid justification for human rights abuses. Consider Japan during WW2 for a concrete example of how far this can go.
Encouraging dissent is a small piece of what is needed. Developers must certainly consider the moral and ethical consequences of their work, but most situations are not going to be black and white, and no matter how diabolical a project might be, there is always a way to rationalize it. What is most needed is widespread public debate among those who understand the technologies in order to form consensus on appropriate directions of development. Any consensus then needs to be effectively communicated to the public and to policy makers. This helps everyone, including developers, make better decisions about what they should or should not support. Forums such as HN and various blogs can foster debate, but there seems to be a need for a more consolidated approach that someone needs to establish. As far as communicating consensus goes, I think that nonprofit organizations like the Electronic Frontier Foundation can be a good conduit, but they need more widespread support from the technical community.
Yea I frequently see sentiments like “if only everyone would stop working for X”. It’s true but when we’re all locked in a game of the prisoners dilemma, moralizing isn’t going to get us to move, laws and changed incentives will. I think americans finally figured that out about climate change, which is why we’re looking at carbon taxes instead of just public pressure.
It's worth remembering that most of the directly attributable harm we associate with tech was created by small entities like the NSO Group and Cambridge Analytica.
Instead of conscientious objectors applying pressure to large companies, it might be better for tech employees who come across small companies doing shady activities to report them. Many times, these smaller firms are more susceptible to pressure anyways, so this is a double benefit.
Being smaller, though, those companies can simply be up front about what they do during recruiting and people who don't like it won't sign up for them. Succinctly, prudes don't work in porn.
As someone who has worked in sf tech for 5-6 years I feel like I’ve gotten to the point where I’ve become too critical of the industry to continue working in it.
I've always ended up with, and still reach the conclusion that there are only two tenable options which will work against a corrupt and unethical system.
1. Infiltrate, subvert, and implode the system from within (that's meant to be as Tzu-esque as it sounds).
2. Using parallel propaganda to get the public to realize that they have the ability to disable the system if they can collectively wield a moral, objective, and ethical compass (with all the McLuhan-esque difficulties that come with it).
I prefer the former. As dangerous as it is, I think it's strategically easier to wield covert action as a tool for effecting change than it is to attempt a unification of the masses. But, maybe a combination of both these approaches would be ideal.
For my part, I'm just not interested in working at facebook or google or certain other companies because I disagree with their practices. Luckily, I'm far enough along in my career that I don't need to take just any job, even if it is limiting to be a privacy conscious data scientist.
I do worry that people like me just staying out of these places leads to an echo chamber, and change from within is great. I wonder whether my individual impact on social good is greater trying to be moral at a moral company, or trying to be moral at an immoral company.
Luckily I don't have to twist my logic around to justify staying and vesting.
I respect the choices of tech sector workers who choose to protest against certain company policies and even refuse to perform certain types of work that enhance mass personalised surveillance and the enhancement of military lethality.
It’s a different world from 30 and 70 years ago.
70 years ago nearly all R&D spent was specifically military in nature. So it was pretty clear from the point of hire that you were(or weren’t working on military or related(surveillance) projects.
30 years ago military and commercial R&D hit parity and it was still quite easy to known going into an employment contract what side of the fence one was on.
Today, almost all R&D is commercial in nature, but with vast duel use military/surveillance potential.
So it’s much more convoluted and nebulous to delineate project output, perhaps a major contributing factor to recent decisions and actions.
Combined with the long standing low supply, high demand for talent enhancing leverage.
I agree that tech talent conscience is critical.
I only hope we see the same from Chinese software engineers working at Baidu, Alibaba, Tencent, Huawei, etc.
I wonder if the future will see a convoluted analog to the Manhattan Project, the Rosenbergs, and apex talent migration out of 1930’s fascist countries.
I remember drafting up a blog post a decade ago where I theorized/ranted that tech workers would end up being the bad guys in history. I didn't post it because it seemed so unrealistic at the time (and because I don't really know what I'll do when tech is killed... maybe some kind of appliance repair?).
What I object to is mass media and political activists trying to shame or force me to agree with their idea of "ethics" and political positions. I'll decide that for myself thank you very much. When you see activists making noise you can bet there are plenty of people who disagree with them. Unfortunately when you have outlets like the New York Times and mobs of Twitter journalists adding fuel to the fire (for one side of the debate only), speaking out for what you believe becomes incredibly dangerous if you don't go with the mainstream message.
So yeah, I object to everything this article says and stands for. Sadly I can't do that under my real name because I need to keep my job.
Discussions about ethical, privacy, surveillance issues often get hand waved and diluted here. See yesterday's thread on Google's retaliation as an example.
We need a diversity of communities that can represent the the software industry in more dimensions than the VC funded side of things, growth hacking and the singular focus on success that are often in heavy conflict with any kind of value dimension and ultimately greed and opportunism crowds out every other concern.
There is also often a extremely selfish mercenary view of the world this is disturbing and can jade most of us and needs some maturity to handle without letting it poison ones worldview.
Right - it's probably very relevant to discussions of, say, Google's AI council that Google's purpose in putting Kay Coles James on their board was not that the Heritage Foundation's views per se should be represented (note that they didn't do corresponding representation of other political factions) but that she and the foundation have significant legitimacy and influence with the politically powerful parts of the American right wing, with which Google is generally in political trouble, and therefore it was a good way to allow Google to execute on its AI-related goals and therefore make money without political backlash.
A genuine ethics board would not include people whose real job is to curry favor with government - it's sufficiently close to unethical that it's a bad way to get started - and would be generally fine with saying "This task is ethically permissible, but not an ethical imperative, and so if it's politically difficult to do it, we should just do other things."
[+] [-] ABCLAW|6 years ago|reply
Tech has a very strong history of conscientious objectors. They're not the problem. Other institutions need to do their part.
[+] [-] mfoy_|6 years ago|reply
Maybe we just, as a society, need to adopt morality and ethics more deeply into our cultural DNA.
[+] [-] jammygit|6 years ago|reply
[+] [-] anarchodev|6 years ago|reply
Further down in this thread the discussion has broadened to worker rights in tech-adjacent jobs (eg. amazon warehouses). It would be great if we could have one big union that fights for the rights of all workers, and if you think so too you should look into joining the IWW.
Yes, our society does need to adopt morality and ethics as guiding principles in our work, and organizing your workplace is not some orthogonal venture but is actually part of that goal.
[+] [-] povertyworld|6 years ago|reply
[+] [-] halfmatthalfcat|6 years ago|reply
[+] [-] mindslight|6 years ago|reply
Traditional engineerings have longstanding concepts of ethics, plenty of professional organizations, and even professional licensure. This does not stop mechanical engineers from adding to USG's weapons disparity or civil engineers from constructing more profit-center cages.
[+] [-] beatpanda|6 years ago|reply
https://www.youtube.com/watch?v=DwbzxemJZIc
[+] [-] currymj|6 years ago|reply
[+] [-] njepa|6 years ago|reply
[+] [-] philpem|6 years ago|reply
I was a student member for a couple of years, but beyond the occasional Government IT job advert which required BCS qualification and accreditation, they seem to be extremely quiet (or not very good at advertising).
[+] [-] msla|6 years ago|reply
Moreover, "coding" is a cross-cutting concern in that a lot of people who aren't formally programmers have been sold on the idea of being able to automate parts of their job by writing code, and I'm not sure how you'd yoke all of those disparate fields (especially academics outside of industry) into a single labor organization.
[+] [-] threatofrain|6 years ago|reply
[+] [-] 3xblah|6 years ago|reply
From the author's website, listed at the bottom.
Imagine if journalists and other people posting to the www called "AI" what it actually is, instead of constantly portraying it as something futuristic to capture people's imaginations.
Even terms like "statistical inference" and "automated decision-making" could probably be explained using more common language to be comprehensible to the general public.
[+] [-] newscracker|6 years ago|reply
In some companies, the culture is also that of being oblivious (or acting oblivious) to the harm caused by the work. Facebook is the best example of this. Has there been any conscientious objector in that company? People like the WhatsApp and Instagram founders, who grew a conscience after getting billions in their pockets, don't count (even if one or two left some money on the table when quitting).
Money tells many a true tale of what goes in in people's minds.
The article, sadly, does not mention Facebook and all the surveillance that it enables and grows, along with oppressing people by forcing them to use "real names".
[+] [-] hnruss|6 years ago|reply
[+] [-] mc32|6 years ago|reply
[+] [-] AstralStorm|6 years ago|reply
[+] [-] eeZah7Ux|6 years ago|reply
[+] [-] int_19h|6 years ago|reply
[+] [-] bentona|6 years ago|reply
I believe that it is dangerous to conflate job choice with conscientious objection.
Everyone should be held accountable for what they spend their day contributing to - this shouldn't be a special case.
Comparing this to a situation in which someone doesn't want to participate in compulsory murder, and in some cases risks being killed themselves, is not helpful.
[0]: https://en.wikipedia.org/wiki/Conscientious_objector#cite_no...
[+] [-] roenxi|6 years ago|reply
But on the other hand, there is a plausible parallel. The reason for strong controls on privacy is because every couple of generations governments tend to do a lot of damage to their own citizens.
If I thought the worst that was going to happen was the risk of being wrongly accused, public shaming or something, then privacy isn't really worth it. I trust law enforcement to be generally correct in assessing the situation, and if you are doing something shameful you may as well be shamed.
Privacy is important because it is one layer of protection against institutions participating in the murder of citizens, seen in Germany, China, Russia in the last generation and China + others this generation. Who knows for next generation? Change is very quick - there is no law of nature that says it won't be an Anglophone country. We can get thingy about whether a conscientious objector has to be the last person pulling the trigger or not, but in practice that distinction is not an important one.
[+] [-] oldjokes|6 years ago|reply
Most of them drive trucks now. A few lucky ones became artists or writers, but mostly a lot of bad outcomes.
One of these failed truck drivers even makes deliveries to the house of the person who forced him out sometimes, a fact which he loves laughing about.
[+] [-] taurath|6 years ago|reply
[+] [-] itronitron|6 years ago|reply
Worth noting that probably everyone here has seen something like this happen. Not sure if it is specific to tech, but some of the egos and pathologies are extreme.
[+] [-] olalonde|6 years ago|reply
[+] [-] stendinator|6 years ago|reply
[+] [-] int_19h|6 years ago|reply
[+] [-] AstralStorm|6 years ago|reply
[+] [-] britch|6 years ago|reply
I think it's fair to say most of the surveillance technology they're buying is not being used to help people.
Do not forget, the Chinese government is, right now, using surveillance technology to imprison Uighurs.
https://www.npr.org/templates/transcript/transcript.php?stor...
[+] [-] unknown|6 years ago|reply
[deleted]
[+] [-] klez|6 years ago|reply
[+] [-] ghda|6 years ago|reply
[deleted]
[+] [-] 1e-9|6 years ago|reply
[+] [-] AndyMcConachie|6 years ago|reply
https://ctsp.berkeley.edu/ethical-pledges-for-individuals-an...
[+] [-] chillacy|6 years ago|reply
[+] [-] lern_too_spel|6 years ago|reply
[+] [-] ETHisso2017|6 years ago|reply
[+] [-] ben509|6 years ago|reply
[+] [-] randomacct3847|6 years ago|reply
[+] [-] learnstats2|6 years ago|reply
[+] [-] oedmarap|6 years ago|reply
1. Infiltrate, subvert, and implode the system from within (that's meant to be as Tzu-esque as it sounds).
2. Using parallel propaganda to get the public to realize that they have the ability to disable the system if they can collectively wield a moral, objective, and ethical compass (with all the McLuhan-esque difficulties that come with it).
I prefer the former. As dangerous as it is, I think it's strategically easier to wield covert action as a tool for effecting change than it is to attempt a unification of the masses. But, maybe a combination of both these approaches would be ideal.
[+] [-] 6gvONxR4sf7o|6 years ago|reply
I do worry that people like me just staying out of these places leads to an echo chamber, and change from within is great. I wonder whether my individual impact on social good is greater trying to be moral at a moral company, or trying to be moral at an immoral company.
Luckily I don't have to twist my logic around to justify staying and vesting.
[+] [-] chriselles|6 years ago|reply
It’s a different world from 30 and 70 years ago.
70 years ago nearly all R&D spent was specifically military in nature. So it was pretty clear from the point of hire that you were(or weren’t working on military or related(surveillance) projects.
30 years ago military and commercial R&D hit parity and it was still quite easy to known going into an employment contract what side of the fence one was on.
Today, almost all R&D is commercial in nature, but with vast duel use military/surveillance potential.
So it’s much more convoluted and nebulous to delineate project output, perhaps a major contributing factor to recent decisions and actions.
Combined with the long standing low supply, high demand for talent enhancing leverage.
I agree that tech talent conscience is critical.
I only hope we see the same from Chinese software engineers working at Baidu, Alibaba, Tencent, Huawei, etc.
I wonder if the future will see a convoluted analog to the Manhattan Project, the Rosenbergs, and apex talent migration out of 1930’s fascist countries.
[+] [-] mrspeaker|6 years ago|reply
It's quite amazing the difference 10 years makes!
[+] [-] malvosenior|6 years ago|reply
So yeah, I object to everything this article says and stands for. Sadly I can't do that under my real name because I need to keep my job.
[+] [-] throw2016|6 years ago|reply
We need a diversity of communities that can represent the the software industry in more dimensions than the VC funded side of things, growth hacking and the singular focus on success that are often in heavy conflict with any kind of value dimension and ultimately greed and opportunism crowds out every other concern.
There is also often a extremely selfish mercenary view of the world this is disturbing and can jade most of us and needs some maturity to handle without letting it poison ones worldview.
[+] [-] saagarjha|6 years ago|reply
Do they? I tend to find that Hacker News often has a lot of discussion of ethics, especially with respect to technology.
[+] [-] geofft|6 years ago|reply
A genuine ethics board would not include people whose real job is to curry favor with government - it's sufficiently close to unethical that it's a bad way to get started - and would be generally fine with saying "This task is ethically permissible, but not an ethical imperative, and so if it's politically difficult to do it, we should just do other things."