[hvocode]

I think the point is that yes, they have always had the power to do these things - but they haven't thus far. We rely on entities with power (companies, governments, people) to exercise restraint in how they exercise it. Those that DO exercise restraint gain trust since exercising restraint demonstrates an understanding of the consequences of not doing so.

What the Apple move is doing is showing that they are willing to relax their restraint. It gets tricky because everyone agrees that the specific goal here is honorable, but the manner by which they are using their power to achieve it is generalizable to areas that are less honorable. Once they are willing to use their power to accomplish one highly honorable goal, it's not a big ask for them to use it for a slightly less honorable goal in the future. Iterate that a few times and you can find yourself in a very bad place. It's the classic slippery slope argument - when you know there is a slope that leads to dangerous places, you need to not ever start down it no matter how righteous the motive is in starting down that path in the first place. There's a reason we have the old saying "the path to hell is paved with good intentions".

The existence of power isn't what matters: it's the intention and willingness to exercise it. Apple is now demonstrating that they have changed their stance in how they choose to exercise their extreme power. That's worthy of scrutiny.

For a concrete example of where I expect this to naturally lead to: instead of a database of child pornography being the source of the hashes to search for, the Chinese government provides a set of hashes of all known digital photos of the Tianenmen square protests of 1989. Does it really seem implausible for a government like China's to NOT use this kind of technology for that purpose? It's not hard to cook up similar examples all over the place.

[jasonlfunk]

Yeah, but they are already doing this for pointless things. They already use facial recognition on all the photos on your phone. That’s what I don’t understand. The only new thing is what they are looking for and their willingness to alert the authorities. The slippery slope argument that this will eventually be used by China to arrest journalists is scare tactics. We have zero evidence that Apple would allow such a thing to happen. And the only thing stopping them is Apple’s word. The fact that they are announcing this should actually give confidence that they aren’t doing it in the shadows for China. They didn’t have to say anything about this. The fact that they should give you confidence that they are respecting your rights, not evidence that they aren’t.

[ique]

Apple has caved to china many many times, a quick googling of it will give you a list.

The difference between the "old" content scanning and the new is indeed that they are now willing to "use" the results of that. Facial recognition was client-side only (or so they said), the results of which never left your phone.

Now they're doing content scanning and sending it to themselves as well as others.

In parallel Apple is starting up a growing advertising business, hiring aggressively and expecting that to be a big part of their future revenue. If they're now "allowed" (by its users) to do content scanning _and_ sharing the results, why wouldn't they use those results for themselves to target you with ads?

[mtrovo]

i think the remote execution with a remote hash database is the key part thay you need to focus. Checking for faces is something that doesn't need much information outside your phone itself.

What Apple is proposing is basically adding a feature to scan any user's phone for a collection of hashes. Even if they say they will only use this for CSAM this sends a strong message to all government agencies around the world that the capability is over there. Maybe for US citizens this doesn't sound dangerous but if I was a minority or a critic of the government on a more authoritarian country I would jump ship from Apple products right away.