top | item 33511016

The most unethical thing I was asked to build while working at Twitter in 2015

921 points| sgk284 | 3 years ago |twitter.com | reply

408 comments

order
[+] AceJohnny2|3 years ago|reply
> And, for the any employees still at Twitter, don’t underestimate the power of a pocket veto.

This is something I've been repeating to some of my younger colleagues.

Engineers aren't really fungible resources, to the extent that these projects require. Ask any manager how easy it is to swap "allocated resources", and they'll probably sigh heavily.

People are afraid that if they don't follow their manager's every request, they will be fired. But remember that hiring is hard, and managers are loath to fire someone they've already spent so much effort finding, hiring, and onboarding. Finding someone else to do it can take weeks, months, or longer! Which in many cases risks killing the project altogether.

Even if you're at the bottom of the chain, as the person who does the actual implementation, you have a lot of power on what gets prioritized.

See also the oft-circulated OSS "Simple Sabotage Field Manual" http://svn.cacert.org/CAcert/CAcert_Inc/Board/oss/oss_sabota...

[+] antognini|3 years ago|reply
Reminds me a little of the story [1] about how in 2005 the execs at Google had a meeting to figure out what to call "Satellite View" in Google Maps. One faction did not like the name "Satellite View" because it was technically incorrect as many of the images had been taken from airplanes, not satellites. But the proposed alternatives like "Aerial photography" all sounded awkward. Right before the meeting ended Sergey Brin decided it would be called "Bird Mode."

Later on when the engineering team was actually implementing it they thought Bird Mode sounded dumb and just called it Satellite View. And so it has been ever since.

[1]: https://twitter.com/btaylor/status/1099370126678253569

[+] Waterluvian|3 years ago|reply
“Pocket veto” is a term I need to remember.

One of my proudest moments was about seven years ago. I was two years into my career as a junior software engineer with no academic background in programming. I was by any measure an impostor and I worked very hard to learn and impress and earn the luck I was given with that job.

A PM, one who I liked and wanted to impress all the time, came to me asking for help to get git commit history for each person on our wider team “to measure how productive everyone is being.”

Despite being anxious about “what-ifs” like being blacklisted or some other concepts I knew nothing of, I gently explained why it would be a bad metric. I remember even saying, “some of the best engineering someone can do is to write negative lines of code.” I felt so wise despite being so green.

He pressed the matter and I calmly said that I said my part and I’ll play no role in this.

I asked around weeks later and apparently he approached nobody else and the issue was dropped.

Maybe this is a mundane anecdote or I’m not telling it properly but I’m still so proud that I was even capable of seeing the ethical dilemma, let alone acting correctly on it. Those years were full of “I have no clue what normal looks like in this industry.”

I feel somewhat confident in saying that experience emboldened me to do the right thing even if it was scary. Sometimes I worry that I fly too close to the sun with my attitude of “you won’t fire me.” But so far it’s worked.

[+] ryandrake|3 years ago|reply
It depends on how junior the engineer is. My first job out of college, I was asked to write some code to cheat a benchmark, basically detect when a particular benchmark program was running and only then put the software into an alternate "fast path" that would result in better benchmark results. I agonized over this and didn't want to refuse. This was my first real job as a professional developer, and I didn't want to make waves. Eventually I got the nerve to tell my boss I was uncomfortable with the assignment, and he said "Oh, no problem at all! We keep our devs happy here." and assigned me onto another task. Joe, three cubicles down was more than happy to write the benchmark-cheating code.
[+] piva00|3 years ago|reply
I have successfully implemented pocket vetoing at the most immoral company I worked for, it was a brief stint (caused by the moral issues) where I could play around not delivering all the features management wanted to gouge their customers by playing with other priorities.

You don't need to do it, you don't even have to explicitly say no, you can just always find (or create) work that's more important to do than breaking your own morals. The worse that can happen is someone else gets the hot potato.

[+] zimpenfish|3 years ago|reply
> managers are loath to fire someone they've already spent so much effort finding, hiring, and onboarding

Caveat: this applies to perms. It doesn't apply nearly as much to contractors (as my many experiences with saying "No, but..." to managers and being canned can attest.)

[+] yodsanklai|3 years ago|reply
> But remember that hiring is hard, and managers are loath to fire someone they've already spent so much effort finding, hiring, and onboarding. Finding someone else to do it can take weeks, months, or longer! Which in many cases risks killing the project altogether.

Anecdata. One of my colleague got fired for not meeting expectations at his level for two consecutive halves. From what I've seen, he was competent and provided value to the project. Some companies have high turnover and are functioning with the idea that everyone is replaceable.

[+] notyourday|3 years ago|reply
> Engineers aren't really fungible resources, to the extent that these projects require. Ask any manager how easy it is to swap "allocated resources", and they'll probably sigh heavily.

I'm hearing Meta, Stripe, Google, Netflix, Lyft and Uber are hiring like crazy for amazing salaries. Not only that but one basically just needs to sort of show up half the time and surf the net 99% of the time there.

That was obviously sarcasm.

[+] alexb_|3 years ago|reply
“We should know when users leave their house, their commute to work, and everywhere they go throughout the day. Anything less is useless. We get a lot more than that from other tech companies.”

This should be posted absolutely everywhere with this as the hook. This type of request and the admittance that companies give even more than that all the time is headline news worthy.

[+] streblo|3 years ago|reply
> Most people don't really appreciate how close Twitter was to shutting down.

> Twitter was on its death bed and was desperate for money.

I worked at Twitter at the same time, and while the company definitely was going through a rough patch at that time, it was absolutely not anywhere close to 'shutting down' or 'on its death bed' financially.

[+] mikeyouse|3 years ago|reply
Yeah this kind of thing is easily verifiable.. Per page 41 in their 2016 annual report, the balance of cash + short term equivalents went from $3.6 billion in 2014 to $3.5 billion in 2015 to $3.7 billion in 2016. Their annual GAAP loss was roughly 1/7th of that. Not the most profitable company in the world, but they had plenty of cash and hand and no real trouble fundraising.
[+] ReptileMan|3 years ago|reply
Never let facts derail a good story.
[+] miiiiiike|3 years ago|reply
Finally, someone who quit.

So many of these stories are from someone who built the thing, profited, left, and then took up a new chapter of their career talking about how everything they did at <BAD COMPANY> was bad and that they should now receive funding, back pats, and NPR airtime for their new <GOOD COMPANY>.

My question is always: "So, are you going to give the money back?"

There really is a middle ground between just following orders and dedicating your life to sabotaging a company from the inside because someone there once thought about doing something that didn't 100% align with your personal mission.

You can refuse and you can quit.

More people need to read books on engineering ethics.

[+] jl2718|3 years ago|reply
> …Elon will do far worse things…

Non-sequitur. The story is about middle management doing evil things for almost no incentive except a small pat on the back for padding a short-term revenue number, while the actual owner-leader who benefits the most shuts it down.

[+] hayst4ck|3 years ago|reply
All individuals are incentivized to do the wrong thing. CEO's are incentivized to sell data to make money. Engineers are incentivized to create bad software via making the people who pay them happier. Users are incentivized to give up their data in exchange for a free service. Politicians are incentivized by political donations and getting information they aren't constitutionally privileged to get.

Doing the ethical thing requires making less money (or losing money) for nearly all parties involved. Doing the right thing requires sacrifice.

In a happy world, the CEO has long term vision and sees the long term cost of loss of trust. The engineers see the ethical problem or betraying their peers and use their pocket veto to do the right thing. The user should be willing to pay a reasonable cost to receive the service they use. Politicians should see that the individual incentives harm the whole and create regulations that disincentivize the poor behavior.

Non-rhetorically: How do we ensure as a society that we live in the latter, and not the former?

[+] rcoveson|3 years ago|reply
I think the only way is to raise a generation of people who see data aggregation/brokerage and user-device-hijacking as immoral, much like how we raised people staring last century to view eugenics as immoral. The weird approach that Stallman has always taken is the only one that can win, as crazy as it seems.

We need religious fervor. We need to decry bundling spyware and "analytics" with free alarm clock apps as evil. Finally, we need "know-it-when-I-see-it" type Software Decency laws that we can leverage to fine evildoers into oblivion (of course, those will follow automatically if we succeed in moralizing the issue).

[+] bogwog|3 years ago|reply
Twitter and tech companies are not the first industry to have ethical problems like this. This is a problem we've solved many times in the past with strict regulations and laws. Finding ethical people is expensive, but writing laws is cheap.

If actual data privacy laws existed in the US, this situation would never have happened. In the linked twitter thread, he says that "legal" said it was ok. That right there is the safety valve that we can control to keep corporations in check.

Why doesn't my local supermarket price gouge us when there's a hurricane about to hit? That's an obvious way to increase profits. In fact, if it weren't illegal to do that, I'd argue that any CEO who didn't do that for "ethical reasons" should be fired and possibly even sued by shareholders.

[+] louthy|3 years ago|reply
That’s what governments and laws are for. Expecting companies to work for the greater good is naive at best; and we shouldn’t really get upset when they don’t. That’s not their motivation, as you’ve highlighted.

Strong legislation and independent legislators are what’s needed

[+] ramesh31|3 years ago|reply
>Non-rhetorically: How do we ensure as a society that we live in the latter, and not the former?

Incentive alignment. Nothing short of hardcore government regulation of personal data, and the remuneration for (opt-in) usage of said data, will change anything.

[+] akira2501|3 years ago|reply
And the market fails to functionally solve any of these problems because it's monopolized by two players who follow the exact same playbook.

These aren't intractable problems, and they don't just come from nowhere.

[+] mmmmmbop|3 years ago|reply
You seem to forget that a law banning exactly this type of behavior does exist in the EU. It's called GDPR, and it's commonly derided on HN.

Just goes to show that governments can be effective in working for the citizens' interests.

[+] ploum|3 years ago|reply
For one story like this which emerges because the engineer refused, how many stories we will never heard about because it was simply done?

As software engineers, we are just like medical experts talking about the toxicity of cigarettes while ourselves buying cigarettes and distributing them to our own children.

[+] imiric|3 years ago|reply
> As software engineers, we are just like medical experts talking about the toxicity of cigarettes while ourselves buying cigarettes and distributing them to our own children.

It's even worse than that. Most of the people working in adtech are actually producing cigarettes, and laughing all the way to the bank. Many of them are on this very site.

https://www.youtube.com/watch?v=tHEOGrkhDp0

[+] xnx|3 years ago|reply
Good reminder that while Google gets a lot of negative privacy attention it is telcos, ISPs, and lesser-known apps that are the most deplorable actors data collection and selling.
[+] toofy|3 years ago|reply
> As far as I know, the project actually got canned. Jack genuinely didn’t like it. I don’t know if this mindset will hold true with the new owner of Twitter though. I would assume Elon will do far worse things with the data.

yeah, this is a major concern of mine now. while a few months ago i had some minor concerns with elon discussing taking it over, his behavior since this started has elevated those concerns to an absolute red alert level. the kind of data he has access to is terrifying.

i’m predicting whatever it is will make the facebook/cambridge analytica thing look tame in comparison.

[+] callamdelaney|3 years ago|reply
Why does he assume Elon will do far worse with the data? The anti-Elon narrative is pretty dull and so far generally unsubstantiated.
[+] Kye|3 years ago|reply
Desperate people do desperate things. He's staring down the barrel of a $1B/year debt service and a collapsing reputation. That reputation is how he convinces investors to keep giving him money while timelines keep getting pushed years into the future. He's the guy who landed two rockets on live TV (oversimplified, but you know reality distortion fields). If he becomes the guy who fumbled Twitter after playing up how good he would be for it, it's over.

This is how "Do No Evil" Google started mining data and buying up the nascent AdTech industry. They weren't mustache-twirling villains, they were desperate to save the company with antsy investors breathing down their necks. They had to do something to justify themselves to investors post-.com bust, and all that data was right there.

[+] tbrownaw|3 years ago|reply
He's a Them instead of an Us, and is therefore evil and motivated by ill intent.
[+] nrmitchi|3 years ago|reply
This really isn't a just for Twitter; this is the danger of selling any application with a large install-base. Doesn't really matter if it's a social network app, borderline-useless mobile app, Facebook App (I'm looking at you, Cambridge Analytica), chrome extension, or pypi/npm module, all of these things are capable of collecting extremely fine-grained user detail, and selling it off.

It doesn't matter if the current owners don't/won't do it, there is essentially nothing that prevents someone else from buying it up, and doing nefarious things with the existing install base.

And as far as "Terms of Service" go, there is essentially nothing to prevent a future owner from updating the Terms of Service, and then doing the above.

[+] xrd|3 years ago|reply
Anyone using social media services need to pay attention to this story. When the profit margins shift ever so slightly, or say massively like with the Apple changes, then these companies will take meetings with executives like this Telco who wanted data on when people are going into their competitors stores. Unbelievable, or should I say, totally believable and totally expected.
[+] woojoo666|3 years ago|reply
> As far as I know, the project actually got canned. Jack genuinely didn’t like it.

> I don’t know if this mindset will hold true with the new owner of Twitter though. I would assume Elon will do far worse things with the data.

When has Elon been against user privacy? Also, isn't Elon good friends with Jack? I feel like they would see eye to eye with this. In fact Elon seems like the type that would try to champion emerging fads like crypto, differential privacy, and zero knowledge proofs. Harvesting data is boring and easy.

[+] FireBeyond|3 years ago|reply
> When has Elon been against user privacy?

Get into a car accident, and want some blackbox data from your Tesla? Good luck - get ready for a lot of legal costs.

Get into a car accident, and it makes Tesla look bad? Tesla will hold press conferences and release your telemetry data to the media, whether you want them to or not. Exceptionally misleading data in some cases - one fatality collision where autopilot was being blamed, Tesla said "Woah, hold up. Not true. Driver was distracted. In fact, the car warned him to put his hands on the steering wheel before the collision!"

In reality, the car had issued -one- warning about the steering wheel, and none after that, and that one warning was -eighteen minutes- before the collision.

[+] woodruffw|3 years ago|reply
Elon has a significantly stronger profit motive, given his high purchase price and Twitter’s otherwise tanking advertising sales.
[+] yewenjie|3 years ago|reply
> Harvesting data is boring and easy.

Boring, easy, yet highly profitable. And sometimes the only boring and easy way to be profitable.

[+] nrmitchi|3 years ago|reply
When "selling user data" is the line between losing 44-billion dollars and NOT losing 44-billion dollars, lines get very blurry.
[+] bpodgursky|3 years ago|reply
Elon has never had a company that made money on ad revenue, despite every other tech CEO on earth trying to monetize in that direction.

So I agree, and feel like the default assumption is that he's going to try to get users to directly pay for content, which is what he's doing with Twitter Blue. We'll see if it works.

[+] scarface74|3 years ago|reply
Twitter has around $1.3 billion in free cash flow not counting the one time settlement they did last year.

Now Musk is on the hook for over $1B in interest payments after buying Twitter and overpaying for it. Do you really think you can trust him to do what’s in the best interest of users?

[+] jiveturkey|3 years ago|reply
> With Twitter's _change in ownership_ last week, I'm probably in the clear to talk about the most unethical thing I was asked to build while working at Twitter.

Generally not true/safe. Any NDA still in effect would be transferred to the new owner. If the author genuinely believes this, they may want to delete this tweet asap. If it's just rhetorical, well ok then.

[+] SilverBirch|3 years ago|reply
Very true about the pocket veto, and I've said this to my team before - I can give advice, I can argue for my values, but my lever as a manager is hire or fire. I don't have time to do the implementation, and the person doing the implementation realistically is going to decide the inplementation. I can influence, but really, they decide.
[+] stephen_g|3 years ago|reply
"We get a lot more than that from other tech companies."

And hence why almost every app on my phone has location access 'never' and only the ones that really need it have it 'while using app'.

Of course, I never even got the Twitter app, I've always just used it in Safari on my phone.