top | item 37384517

When tech says ‘no’

223 points| mooreds | 2 years ago |ben-evans.com

328 comments

order
[+] colmmacc|2 years ago|reply
I spent a few years in Ireland and the Netherlands lobbying against insecure voting machines. I had so many conversations with politicians and civil servants where these effects were in abundance. In each country I was representing a group that included the most eminent and experienced computer scientists in the country. Whenever we briefed someone for the first time, it would only take a minute or two to cover how without a voter-verified paper audit trail, no-one knew how to build a system that provided anonymity, verifiability, and resistance to voter coercion and vote selling. But it just never clicked, so many of them refused to believe that we couldn't simply "nerd harder". And of course, there was no shortage of charlatans who would tell them that they could solve it.

In Ireland, we put enough pressure on the politicians for them to create a cross-body commission to investigate. Because anyone could make detailed submissions, and because the commission treated these submissions like the clerks of the Supreme Court treat amicus briefs; it was pretty effective. The commission ended up pausing, and ultimately abandoning, the rollout. Ever since I've learned to appreciate any avenues to "de-politicize" a controversy and get it to that kind of body.

[+] jackgavigan|2 years ago|reply
I would argue that there's a fourth kind of ‘no’, when tech decides that enough is enough, and says "No, fuck you."

We've reached that point in the UK, where the government has proposed draconian legislation[1] that would allow the government to force companies to create backdoors into encrypted messaging services.

As a result, Whatsapp[2] and Signal[3] have said that they will pull out of the UK, and Apple[4] has said it will remove Facetime and iMessage from the UK if the legislation passes.

1: https://www.eff.org/deeplinks/2023/07/uk-government-very-clo...

2: https://www.theguardian.com/technology/2023/mar/09/whatsapp-...

3: https://www.bbc.co.uk/news/technology-64584001

4: https://www.bbc.co.uk/news/technology-66256081

[+] Ensorceled|2 years ago|reply
There is kind of a "boy who cried wolf" thing going on where the tech industry has pushed back hard on pretty much any sort of regulation and oversight that the regulators automatically assume "they’re saying no because they just don’t like it.".
[+] AnthonyMouse|2 years ago|reply
A big part of the problem here is that most of the proposals to regulate tech companies actually are terribly crafted attempts to regulate something the drafters poorly understand, or naked power grabs by other industries (legacy media conglomerates being a common perpetrator).

A good heuristic is to look at which part of the industry is opposing it. If it's opposed by small businesses or individual developers or civil liberties organizations, that's a bad rule even (or especially) if huge tech conglomerates like it. But if it's the reverse -- like antitrust enforcement -- now you might be onto something necessary.

[+] timmaxw|2 years ago|reply
> pushed back hard on pretty much any sort of regulation and oversight

I think there's a selection effect going on here: If tech and politicians agree that some policy is a good idea, then tech does it voluntarily, so no regulation is needed. The only cases where it becomes a matter of regulation are the cases where tech and politicians disagree.

For example: Remember the privacy discussion around COVID-19 exposure-tracking apps? If the exposure-tracking apps had been implemented in a naive way, they would have been incredibly invasive to privacy. But tech proactively figured out good solutions to the privacy questions, so it never became an issue. If some politician _had_ proposed regulation saying that exposure-tracking apps needed to protect privacy, then tech wouldn't have pushed back, because that's what they were already doing anyway. But because tech was already doing it, politicians didn't propose the regulation.

So, because an issue never becomes a matter of regulation unless tech pushes back on it, it ends up looks like "tech pushes back on all regulation".

Furthermore, in the cases where tech and politicians disagree, the politicians haven't always been right. For example, GDPR cookie banners are a joke. California's AB5 law is another example, as the original article mentioned.

So, I don't think "boy who cried wolf" is a fair analogy. Tech companies aren't always right, but it's not as if they're automatically opposed to all new policies; and when they do oppose politicians' proposed policies, it's sometimes for good reasons.

[+] fallingknife|2 years ago|reply
And for what proposed regulation of the tech industry was the pushback unwarranted?
[+] naetd|2 years ago|reply
The first type of 'no' is summarized as just being annoying to implement and not making a big difference, but I think it can be a lot deeper and more consequential than that.

There can be regulation that will damage a companies profits, but also provides positives to public health or other beneficial outcomes. Deeply profitable companies will fight tooth and nail against these regulations even if they are full aware of the damage they are causing. They will come up with as many convincing sounding reasons to say "no" as possible in the name of those immense profits they enjoy, and use techniques like expensive lobbying, sponsoring pseudoscientific studies, running ads, play up fears about economic damage or other negative outcomes of the policy, etc. They try to make it sound like the second or third kinds of "no" in the article and paint it as a bad idea, or impossible to do, or anything else they can to prevent the regulation. And if a certain individual at that company doesn't want to fight for their unethical profit, they'll be swiftly replaced with someone who will.

The obvious (non tech) example is something like the tobacco industry, which spent millions on manipulating public and policy opinion using misleading scientific sounding language or studies to prevent or delay regulation despite being fully aware of the many health detriments of smoking. Public health has been significantly improved as a result of smoking reduction, restrictions on where you can smoke in public spaces, age restrictions, whatever.

I think there is a lot of this currently in companies profiting off social media, oil and gas, and selling user data.

[+] AnthonyMouse|2 years ago|reply
> Deeply profitable companies will fight tooth and nail against these regulations even if they are full aware of the damage they are causing.

Unfortunately this is also very similar to one of the most insidious forms of regulation -- the mildly inefficient requirement. You have something which is absolutely not going to bankrupt the company, but it costs three times more than it's worth.

It may even provide some benefit to someone -- someone who is happy to lobby in favor of it if it means they get a third of the money that it's costing customers to require it.

But then inefficiency increases and costs go up and barriers to entry to go up and the market becomes more concentrated, and the incumbents only make a weak showing of opposition because it's not going to kill them and they actually like that it might kill some of their smaller competitors.

So rules like that accumulate, even though they're each a net negative to the world, until people can't make ends meet because everything costs so much more than people get paid. And nobody can point to one single rule as the problem because it's really ten thousand of these little inefficiencies adding up.

[+] FabHK|2 years ago|reply
> Deeply profitable companies will fight tooth and nail against these regulations even if they are full aware of the damage they are causing.

There was an article recently about health labels on food in Mexico. The food manufacturers were not allowed to print cute mascots on the packaging for certain foods aimed at children, and had to place a warning label on certain foods. In the first case, the manufacturers switched to transparent packaging, and printed the mascot on the food itself (so that it was clearly visible). In the second case, the manufacturers basically made the front and the back of the packaging the same, but put the mandatory warning only on one side (so store employees would put it on the shelves with the warning on the back). I wish they would pursue ways to make food healthier with the same energy.

ETA: https://www.schneier.com/blog/archives/2023/08/hacking-food-...

https://news.ycombinator.com/item?id=37245593

[+] lifeisstillgood|2 years ago|reply
I think there is a good ole 2x2 grid here.

The socially positive nature of the industry on one axis and the degree of strong completion on the other.

So we can have say Retail food stores, sainsbury's and tesco, that deliver mostly positive things (food!) in a highly competitive manner.

We can also see positive industries (water / sewage) that have terrible competitive landscapes (fundamentally monopolies/ utilities). These need to be regulated differently - ie with hands firmly clasped round the throat of all participants.

Bad industries and bad competition looks like the illegal drugs trade (I personally think the cut throat nature of retail stores is as literal cut throat as we want. When the completion stops focusing on making the product better and starts focusing on killing the other stores employees we are not seeing improved markets

And your example was bad industry / good competition- cigarettes are a good example here.

I think it's worth adding a third dimension to the grid - time and future shape. The retail food model is a good one but over time we can see the effect on out of town car parks, the urbanisation vs wqlkability etc etc. Intervening in how stores advertise the price of milk won't help this. But neither will "nerd harder" - there is no solution to "this business model if continued will go the wrong way" that does not involve chnaging the business model - ie charging for car parking space or something.

Anyway, it struck me as a useful simple graph. As business models move to different parts of grid they get regulated differently, and adding time/dependencies in means we can shape the results.

But in the end I am arguing for smart proactive interventionist government.

Let governments be governments

[+] nvm0n2|2 years ago|reply
> also provides positives to public health or other beneficial outcomes

But the people asserting these positives are also lobbying, making convincing sounding arguments, running ads, playing up fears, sponsoring pseudo-scientific studies and all the other ills you criticize. And they'll do that even if they're fully aware of the damage they're causing, or will cause with their proposals.

[+] mdgrech23|2 years ago|reply
100% - people don't realize what kind of mastermind bullshit companies come up with to keep the profits rolling in nevermind the damage caused.
[+] nickelpro|2 years ago|reply
> There followed a desperate scramble to exempt over 100 professions, from doctors to truck drivers to hairdressers, before the whole thing had to be abandoned. A lot of people told the politicians about the problem, but the politicians just said “everyone always says every law will be a disaster” and ignored them. Oops.

AB 5 passed, along with its long list of exceptions, and is law in California right now. It wasn't "abandoned" in any sense. Getting facts right is important to a persuasive argument

[+] larsiny|2 years ago|reply
"The three companies, now also joined by Instacart and Postmates, funded a ballot initiative, Proposition 22, to exempt both ridesharing and delivery companies from the AB 5 requirements, while also giving drivers some new protections, including minimum wage and per-mile expense reimbursement. Proposition 22 passed in November 2020 with 59% of the vote.[8][9]"

From: https://en.wikipedia.org/wiki/California_Assembly_Bill_5_(20...

So the law has an exemption for the exact workers it was targeting.

[+] csours|2 years ago|reply
Disclosure, I work for GM.

Type 1: The tradeoffs are not in my favor.

Type 2: You have not understood the tradeoffs.

Type 3: No one can evaluate the proposal.

> "When policy-makers ask for secure encryption with a back door, we do not always see that this would like be telling Ford and GM to stop their cars from crashing, and to make them run on gasoline that doesn’t burn. Well yes, that would be nice, but how? They say ‘no’? Easy - just threaten them with a fine of 25% of global revenue and they’ll build it!"

> "This would like be telling Ford and GM to stop their cars from crashing"

Easy - car doesn't go until all seatbelts are on. All seats face backwards. Helmet and HANS device for all occupants. Maximum speed is 45 mph. Cars are wrapped in giant foam pads. Cars are limited to roads mapped by the automaker. (Type 2b [I have interpreted the intention of your Type 3 proposal] - Customers would revolt)

> "Make them run on gasoline that doesn’t burn"

Easy - catalyze gasoline to hydrogen and use a fuel cell. Well not easy, but possible. (Type 3 - I can propose this, but no one can evaluate it without doing a LOT of work)

====

Personal opinion: People feel like experts have lied to them, because experts have lied to them. We can't trust experts. How should people think? Specifically when it is very expensive to test something? 'Expensive' includes all kinds of risk, not just money spent. 'Test' includes "what will this do to me?"

So some people think that there is a 100 mpg water carburetor that Shell bought the patents to and maybe the inventor had an 'accident'. In reality, (Type 2) They have not understood the tradeoffs. 100 mpg is easy - on a speed limited motorcycle on a chosen route. Water carburation is not too hard for motivated and handy person to use. It IS too hard to put on a general consumer's vehicle.

Jet airplanes used to inject water into the engine to get more performance on takeoff. Someone realized that it's cheaper, easier, and saves weight to just inject more fuel. The fuel doesn't fully burn, but it adds to the thrust just by being mass that goes out the end of the engine.

[+] Barrin92|2 years ago|reply
>“Work it out” is generally a demand to invent new mathematics, but sadly, mathematics doesn’t work like that

The article invalidates a somewhat reasonable point by saying this sort of thing. A cryptographic end-user application is not "mathematics". It's a piece of software, running on a piece of hardware, and there is no platonic, infallible security going on. This is the sort of 'no' people utter if they have ideological objections disguised as technological ones.

In reality any system, including cryptographic ones exist on a curve. Differentiated systems for access exist. The honest criticism would be that a system with a backdoor is less secure, but it's certainly possible to enable privileged access to third parties while excluding others, it's just riskier. But risk in reality also exists with any encrypted application, because it runs with keys stored on a phone, not in some untouchable maths dimension.

How much you move between access and security is absolutely a question of policy and architecture, not some theoretically impossible thing.

[+] ChrisMarshallNY|2 years ago|reply
This was a great essay.

I really liked his summary:

> A Californian optimist would say that we’ll age out of this. The policy class that got their staff to print their emails will age out and be replaced by the generation that grew up sending emojis, and understands that tech policy is just as nuanced, complex and full of trade-offs as healthcare, transport or housing policy. A European would ask how well California handles healthcare, transport or housing.

[+] dmurray|2 years ago|reply
> Your MPs’ WhatsApp group can be secure, or it can readable by law enforcement and the Chinese, but you cannot have encryption that can be broken only by our spies and not their spies. Pick one.

It doesn't seem technically infeasible for WhatsApp to move to a protocol where, say, every message is transmitted twice, once encrypted with the recipient's public key and once with the NSA's public key. Or for the state to ban all messaging systems that don't follow that protocol.

Arguments against encryption are generally philosophical rather than purely technical.

[+] benedictevans|2 years ago|reply
I am not a cryptographer, but the standard objection to this is that the NSA key will leak, either generally or be stolen by a Russian/Chinese agent. And in implementation, how many keys are we talking about? USA, UK, France, Germany, USA, Australia... Every country's law enforcement will demand a key, and how long will that remain secure?
[+] colmmacc|2 years ago|reply
Modern messaging protocols, including the Signal Protocol used by WhatsApp, use Diffie-Hellman key agreement for Forward Secrecy. DH requires an exchange between two active parties, who will then agree on an ephemeral session key. Ideally the session key is deleted once it is no longer being used, rendering any captured cipher texts useless.

While we could encrypt sessions keys under an escrow key that the authorities control, that's a very serious degradation of forward secrecy. If an authority's escrow key is ever compromised, then all sessions encrypted with keys escrowed are also compromised. Non-negotiated keys that are re-used are also inherently more vulnerable to cryptanalysis, so it's an invitation for trouble if any cryptographic weaknesses are found in the escrow scheme. These are technical considerations.

[+] lstamour|2 years ago|reply
You’re making the original point - you can have encryption that can include the NSA, but it would also end up including other spies too, as secrets tend to leak the more important and widely used they are. The original point was not that you can’t have a secure leak, but that the secure leak ultimately wouldn’t stay secure forever. You don’t want to build the weapons your enemies end up using against you, and in digital ecosystems it’s often trivial to do this.
[+] Sniffnoy|2 years ago|reply
I feel like it's worth linking here to the big report from a few years ago: https://dspace.mit.edu/bitstream/handle/1721.1/97690/MIT-CSA...

Probably the most relevant part from the summary is:

> Third, exceptional access would create concentrated targets that could attract bad actors. Security credentials that unlock the data would have to be retained by the platform provider, law enforcement agencies, or some other trusted third party. If law enforcement’s keys guaranteed access to everything, an attacker who gained access to these keys would enjoy the same privilege. Moreover, law enforcement’s stated need for rapid access to data would make it impractical to store keys offline or split keys among multiple keyholders, as security engineers would normally do with extremely high-value credentials. Recent attacks on the United States Government Office of Personnel Management (OPM) show how much harm can arise when many organizations rely on a single institution that itself has security vulnerabilities. In the case of OPM, numerous federal agencies lost sensitive data because OPM had insecure infrastructure. If service providers implement exceptional access requirements incorrectly, the security of all of their users will be at risk.

[+] dataflow|2 years ago|reply
> we can make it secure, or we can let law enforcement have access, but that means the Chinese can get in too

How is this in category 3? Give decryption keys to your own government and not the others. I get it if you think it'll require more opsec from your government than you might trust them with, or that it'll have other negative effects (see: category 2), but how is this physically impossible (category 3)?

[+] H8crilA|2 years ago|reply
You must have never heard of intelligence agencies, and spying in general. People vastly underappreciate the value that intelligence gives and the lengths to which they'll go just to get access to information. This is as old as humanity, for example Sun Tzu clearly wrote that useful spies are the best compensated employees in the entire government.
[+] benedictevans|2 years ago|reply
If you give the key to every government that demands it (USA, UK, France, Germany, Japan... ), and every agency (CIA, NSA, FBI, DHS, DEA...) then how long will that remain secure? The key will leak and then you have no security.
[+] falcor84|2 years ago|reply
I would assume that the issue is that the mere idea of decryption keys that can be "given" means there is no end-to-end encryption.
[+] elpool2|2 years ago|reply
There are different degrees of “secure”, right? Maybe you could give a key to the FBI without China getting ahold of it, maybe it’s still “secure enough”. But you can’t say it’s “just as secure” as not giving it to them. And that’s what law enforcement often asks for: Give us access without making it any less secure.
[+] bee_rider|2 years ago|reply
I think it must be category 2, given that you’ve gotten a bunch of very strenuous type 2 objections, and no type 3 ones, in the comments they have responded to you.

Actually I think there are almost never type 3 objections. Almost every law is something of the form: “Do this, pay fines, or stop providing your service here.” Of course, the “do this” might be impossible, but there’s no mathematical contradiction in the idea that a company can be run out of business.

[+] csours|2 years ago|reply
It's type 3 because there's also a requirement that there are NO unintended circumventions to security. That sounds like "hard to circumvent", but it's not the same.
[+] bee_rider|2 years ago|reply
I think the third type of “no” just sounds a lot more confusing or contradictory to the engineers and programmers who are tasked with trying to implement a solution that both fits the requirement, and allows the company to continue doing business mostly they were before.

Of course, on the other side, the full command is “change how you provide your service in this way that we’ve specified, or stop providing it.” There is no mathematical contradiction or impossibility here, they just don’t mind if they yank away your livelihood, don’t let them off the hook by assuming they are stupid (they might be, but they probably have someone clever who can feed them enough car analogies…).

[+] eloisant|2 years ago|reply
Maybe stop listening to Big Tech companies and start listening to NPO like EFF or ISOC?
[+] light_hue_1|2 years ago|reply
Some of these show a misunderstanding of the situation.

> Most of the Canadian tech and indeed media industries pointed out how stupid this was, and Google and Meta said that given the choice, they’d stop letting news appear rather than pay a fee they could not control and that had no economic basis. The government thought this was the first kind of ‘no’ and a bluff, but actually, it was the second kind. Oops.

This was part of the intent. It was not a mistake. The goal of the Canadian government has always been to build up Canadian media. Keeping out big foreign companies is in line with that.

[+] paulddraper|2 years ago|reply
Sort of a "oh no! well anyway..." sort of situation
[+] tomComb|2 years ago|reply
The move away from globalization has had an unfortunate side effect: greater political corruption, where local politicians enact laws for the benefit of their local corporate buddies, and pretend (usually successfully) that it is an act of nationalism and standing up to big tech.

That’s what happened here in Canada with Bill c-18, as mentioned in the article. It’s been rather sickening to watch the government defend this bizarre law that is little more then a shake down for their buddies at bell and Rogers.

[+] Apocryphon|2 years ago|reply
This has always followed protectionism, I'd guess. Really the solution would have been to socialize the domestic losses caused by globalization (retraining workers, etc.), but there seems to be no interest or political will towards doing that.
[+] lifeisstillgood|2 years ago|reply
Who are bell and rogers? Some sort of newspaper firm? Plus while corruption is possible, protectionism is the less greasy term, and equally likely to be driving things
[+] rhaway84773|2 years ago|reply
BS. “Tech” as described here says no for one reason and one reason only. It will lose them money. Sometimes it will lose money over the short term. Other times over the long run.

That’s the only reason “tech” says no.

Individuals may say no for some of the reasons mentioned. But this article is basically describing companies and yes, they say yes if they think they will make money and no if they will lose money.

And this isn’t even a bad thing. Companies as a legal fiction exist for the purpose of making money.

What is a bad thing is people thinking that companies (especially public for profit ones) act on any basis other than whether they will make money or not.

Which is why this article is an f’ing disaster and naive beyond the extreme.

[+] ben_w|2 years ago|reply
Back at university, one of my lecturers had a story about having to convince a company to cancel a contract they'd signed with a different lecturer, call them Bob, because the software Bob had agreed to write had provably impossible performance characteristics.

I forget the details, I think it was not quite as bad as "O(1) sorting for any length list" (not even 100% sure they actually told us the specifics) but it was something along those lines.

[+] Doches|2 years ago|reply
Are Apple, Meta, and Signal saying “no” to the UK’s demand for secure-to-GCHQ backdoors in E2E purely to defend their margins? Of course not; they’re saying “the laws of mathematics trump the laws of the United Kingdom”. Apple isn’t promising to pull iMessage (a huge part of their platform lock-in!) from the UK because they want to avoid the expense of complying with this law. Even a cynic would say that they’d love to comply, if only there was a way.

But they literally can’t.

[+] jjj123|2 years ago|reply
How is it that none of these are “no, because that hurts our bottom line even if it’s good for the public”?

The framing in this article suggests that corporations and societies are usually aligned in their interests. This has not been my experience.

[+] edgarvaldes|2 years ago|reply
>First, and this is the default, they’re saying no because they just don’t like it.

>Second, though, the tech industry (or the doctors, or the farmers) might be saying no because this really will have very serious negative consequences that you haven’t understood.

>If the second kind of ‘no’ is ‘that’s a really bad idea’, the third kind is ‘we actually can’t do that’.

I think a big industry is plagued by the first case. On the other hand, a specific tech worker is more prone to say no because of 2 and 3.

[+] cushychicken|2 years ago|reply
Where's the kind of "no" that means "that doesn't align with our business model, so we'll fight it tooth and nail"?
[+] hyperpape|2 years ago|reply
IMO, it’s option (1) if the effect is small enough. Option (2) if there are serious knock on effects for society.
[+] isaacremuant|2 years ago|reply
Don't worry lads, there's always "consultations".

It's always so buteocratic and processy that we can pretend it functions like a democracy, where the will of the people is what matters, and not the will of lobbyists and elite class politicians who've never had to deal with rent prices and economic struggles.