An open letter against Apple's new privacy-invasive client-side content scanning(github.com)
github.com
An open letter against Apple's new privacy-invasive client-side content scanning
https://github.com/nadimkobeissi/appleprivacyletter
430 comments
Because it’s based on machine learning, why can’t some of the network and weights used for the hash be shared? So we can be sure it is not possible to match anything else than children.
The sub narrow network used for detecting only CP is of course secret, but then we know it can’t be used for revealing pictures of police, activists etc
The sub narrow network used for detecting only CP is of course secret, but then we know it can’t be used for revealing pictures of police, activists etc
How to complain: don't buy an iphone next time you need a new phone
Yesterday's discussion: https://news.ycombinator.com/item?id=28085632
I was actually looking to buy the new iPhone should they add back TouchID, but this has been quite the turnoff.
Are Pixel phones the only android phones that allow you to do secure boot with non stock images and basically long term cripple phone functionality once its unlocked?
Are Pixel phones the only android phones that allow you to do secure boot with non stock images and basically long term cripple phone functionality once its unlocked?
I was about to buy M1 Macbook. No thanks.
From the letter:
> Apple Inc. issue a statement reaffirming their commitment to end-to-end encryption and to user privacy.
Apple has no commitment to end to end encryption or user privacy; the premise of this letter is incorrect.
https://www.reuters.com/article/us-apple-fbi-icloud-exclusiv...
The data in iCloud is for the most part effectively unencrypted. iCloud Backup serves as a backdoor to e2e messaging, and iCloud Photos aren't e2e at all. If you are using iCloud Photos or iCloud Backup today (on by default and enabled on most iPhones), you are already uploading all of your data to Apple presently in a way that both Apple and the USG can read without a warrant.
This is by design.
> Apple Inc. issue a statement reaffirming their commitment to end-to-end encryption and to user privacy.
Apple has no commitment to end to end encryption or user privacy; the premise of this letter is incorrect.
https://www.reuters.com/article/us-apple-fbi-icloud-exclusiv...
The data in iCloud is for the most part effectively unencrypted. iCloud Backup serves as a backdoor to e2e messaging, and iCloud Photos aren't e2e at all. If you are using iCloud Photos or iCloud Backup today (on by default and enabled on most iPhones), you are already uploading all of your data to Apple presently in a way that both Apple and the USG can read without a warrant.
This is by design.
Is it ok nowadays to use github for pamphlets, "open letters" (that aren't letters), and other social media and polemic content to protest against privacy invasion of all things?
I'm not usually the most idealistic when it comes to FOSS, but it's stories like this and Apple's stance of "trust us, we promise we know what we're doing and that nothing is going to go wrong" that makes me think all software should be legally required to be open source.
Google Photos and Gmail openly and heavily scan server-side, and I'm sure a lot of us use them, how do we reconcile that?
Honestly I'm glad to see a non-insignificant amount of people in tech take this seriously, especially when the goal Apple announces appears to be for the greater good. It can be hard to stand on the side that doesn't immediately appear to be correct. We have already lost so many freedoms for 'national security' and other such blanket terminology.
Just be warned, there will be those that unfairly try to cast this as helping the distribution of CP. Expect a headline tomorrow: "Edward Snowden et al Supports Child Porn" - or some other hot take.
A few other things:
1. Vote with your feet - Put your money in the pockets of the people aligned to your values.
2. Actively support projects you align with - If you use some open source hardware/software, try to help out (financially or time).
3. Make others aware - Reach out to those around you and inform them, only then they can make an informed choice themselves.
Just be warned, there will be those that unfairly try to cast this as helping the distribution of CP. Expect a headline tomorrow: "Edward Snowden et al Supports Child Porn" - or some other hot take.
A few other things:
1. Vote with your feet - Put your money in the pockets of the people aligned to your values.
2. Actively support projects you align with - If you use some open source hardware/software, try to help out (financially or time).
3. Make others aware - Reach out to those around you and inform them, only then they can make an informed choice themselves.
Don't support Apple! *You must sign up for a Microsoft GitHub account to sign our petition.
While I like the sentiment of protest, I don't understand this logic of putting it on a closed platform of one of Apple's biggest competitors who also doesn't respect your privacy.
While I like the sentiment of protest, I don't understand this logic of putting it on a closed platform of one of Apple's biggest competitors who also doesn't respect your privacy.
I’m not sure what to think of the backlash here. I’m sure that people aren’t trying to minimize the evil of child pornography and exploitation. But anything that has the potential to stop or slow it should be fairly considered. People complaining that Apple is scanning your photos, they are already doing that. Where was this backlash when they released the memories feature? Why is scanning your photos for good pictures of your dog different that scanning your pictures for exploitive pictures of children? Is the problem that they could then share that information with the authorities?
But of course this feature could be abused. But Apple already has all the power it needs to be abusive. They can push whatever software they want to your devices. They can remotely lock and wipe all your data. They already have the power to do all of these things. This statement is simply an announcement that they will be using some of this power to try to stop one of the worst evils in our world. Until they prove that they will abuse this power, I suggest that we let them try.
But of course this feature could be abused. But Apple already has all the power it needs to be abusive. They can push whatever software they want to your devices. They can remotely lock and wipe all your data. They already have the power to do all of these things. This statement is simply an announcement that they will be using some of this power to try to stop one of the worst evils in our world. Until they prove that they will abuse this power, I suggest that we let them try.
Related but slightly off-topic: am I the only one that thinks more technology is not the answer to catching crooks?
Can’t the police do good old fashioned police work to catch people doing these things?
Why does EVERYONE need to be surveilled for the 0.01% (or less?) who don’t behave properly.
To further this point: why do we need cameras on every street, facial recognition systems and 3-letter orgs storing huge data silos and maintaining lists etc etc…
One thought: is it because over 10, 20, 30+ years the police have been de-funded everywhere and become useless for difficult cases and/or militarised to deal with peaceful protesters instead?
Genuine questions.. I just don’t think this surveillance nightmare is the answer, and police could still catch crooks before the internet so what’s the problem.
Most of the people see the adversary in Apple (or governments), I think there is something else:
What about Adversarial Attacks.
Let’s assume someone is going to spread regular memes modified as Adversarial examples to generate the same neural hash as the true bad images.
Thinking back at the political campaigns, these could spread very easily among some voters for some party.
Suddenly you have a pretty serious attack on people of some political spectrum (or whatever group you can target with this)…
I seriously think this wasn’t fully thought through.
Horrible. Apple has every right not to facilitate child exploitation, your rights be damned. Don’t like it? Don’t buy an iPhone.
end of story.
end of story.
I think the important question is, can we trust a large company like Google, Apple or Intel again in the future, and how do we know that they're not spying on us?
At this stage it doesn't matter if it is carefully calculated effort to gradually strip off people from the very sense of any privacy with longing effects on a society (like I described here [1]) or it's some company trying to legalize Spyware Engine for it's own convenience covering it with some story to justify it. Usually they use children for such cover story because people have direct emotional response to that topic and thus stop thinking for just enough time to miss the important issue.
The important issue here is an attempt of Spyware Engine installation/legalization on personal device.
I think this is much more serious than many people might see it at the moment. I think it is attack on a very fabric of free democratic society where human rights have real meanings. It is done by people who do not share those values or not familiar with them or simply do not care about them or even worse - doing it on purpose.
Many can think of their personal device as their home. Some can even think of it as extension of their mind. It is very personal space and attempts to invade it can/should be considered not less serious then invasion into your private home.
Even critics can admit that your personal device indeed can be in your private home space and connectivity of such personal device should not automatically mean that someone or something like AI under control of someone should access it and spy on you without warrant.
Companies are doing shitty things for some time now and they are getting away with them. This can be a reason why they move forward with unacceptable practices of surveillance. Perhaps those companies do not understand that surveillance is completely incompatible with free democratic societies respecting human rights and there is no way they can succeed in combining two incompatible things unless they wish to repeat China way of doing thing. And I believe the later would not be met gladly I even think not so much possible without using firearms to which people would resist with the same effort like they did many times in history to protect own freedom.
At this stage I think it is much more serious than just writing a letter.
There should be a law protection of your personal computer from any Spyware Engine/Agent for any reason other then warrant to avoid bigger and possibly deadly confrontations. Just like your home as your personal space should be protected from Search without warrant in Fourth Amendment [2]. Your personal device is much more personal then your home in a way and deserve to be guarded accordingly. I am not a lawyer and perhaps Fourth Amendment is already enough but then it should be used properly to prevent Spyware Engines in personal devices.
* Amendment IV
The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no warrants shall issue, but upon probable cause, supported by oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized. * [2]
The Fourth Amendment originally enforced the notion that “each man’s home is his castle”, secure from unreasonable searches and seizures of property by the government. It protects against arbitrary arrests, and is the basis of the law regarding search warrants, stop-and-frisk, safety inspections, wiretaps, and other forms of surveillance, as well as being central to many other criminal law topics and to privacy law. [2]
[1] https://news.ycombinator.com/item?id=28084578
[2] https://www.law.cornell.edu/constitution/fourth_amendment
The important issue here is an attempt of Spyware Engine installation/legalization on personal device.
I think this is much more serious than many people might see it at the moment. I think it is attack on a very fabric of free democratic society where human rights have real meanings. It is done by people who do not share those values or not familiar with them or simply do not care about them or even worse - doing it on purpose.
Many can think of their personal device as their home. Some can even think of it as extension of their mind. It is very personal space and attempts to invade it can/should be considered not less serious then invasion into your private home.
Even critics can admit that your personal device indeed can be in your private home space and connectivity of such personal device should not automatically mean that someone or something like AI under control of someone should access it and spy on you without warrant.
Companies are doing shitty things for some time now and they are getting away with them. This can be a reason why they move forward with unacceptable practices of surveillance. Perhaps those companies do not understand that surveillance is completely incompatible with free democratic societies respecting human rights and there is no way they can succeed in combining two incompatible things unless they wish to repeat China way of doing thing. And I believe the later would not be met gladly I even think not so much possible without using firearms to which people would resist with the same effort like they did many times in history to protect own freedom.
At this stage I think it is much more serious than just writing a letter.
There should be a law protection of your personal computer from any Spyware Engine/Agent for any reason other then warrant to avoid bigger and possibly deadly confrontations. Just like your home as your personal space should be protected from Search without warrant in Fourth Amendment [2]. Your personal device is much more personal then your home in a way and deserve to be guarded accordingly. I am not a lawyer and perhaps Fourth Amendment is already enough but then it should be used properly to prevent Spyware Engines in personal devices.
* Amendment IV
The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no warrants shall issue, but upon probable cause, supported by oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized. * [2]
The Fourth Amendment originally enforced the notion that “each man’s home is his castle”, secure from unreasonable searches and seizures of property by the government. It protects against arbitrary arrests, and is the basis of the law regarding search warrants, stop-and-frisk, safety inspections, wiretaps, and other forms of surveillance, as well as being central to many other criminal law topics and to privacy law. [2]
[1] https://news.ycombinator.com/item?id=28084578
[2] https://www.law.cornell.edu/constitution/fourth_amendment
So, if I understand correctly, the NCMEC provides a bunch of hashes of CSAM for Apple to match. This way Apple doesn’t get exposed to the content of images themselves? Then Apple will provide a user’s details plus the IDs of matching content. This identifies the direction of travel for any CSAM content?
So, now NCMEC and any local NCMEC can provide new hashes and identify – possibly even historically – the epicentre of the distribution of a group of images.
Except, if Apple only gets the hashes, what’s to stop a bad actor in a NCMEC from providing non CSAM images to Apple for other purposes?
Seems like this technology should be illegal without a warrant. It’s a wire tap.
So, now NCMEC and any local NCMEC can provide new hashes and identify – possibly even historically – the epicentre of the distribution of a group of images.
Except, if Apple only gets the hashes, what’s to stop a bad actor in a NCMEC from providing non CSAM images to Apple for other purposes?
Seems like this technology should be illegal without a warrant. It’s a wire tap.
It's really hard to not want to throw some Schadenfreude on those who pay mini-bar prices for technology.
Sadly I doubt this will be a last straw for most, given their investment in time standing in lines.
This letter doesn't really read correctly. If you're going to write an open letter, this might be better worded.
>Apple's proposed technology works by continuously monitoring photos saved or shared on the user's iPhone, iPad, or Mac.
It does a check if it's being uploaded to iCloud Photos. It is not (currently, at least) continuously monitoring photos saved or shared; shared is Messages specific for child accounts.
>Because both checks are performed on the user's device, they have the potential to bypass any end-to-end encryption that would otherwise safeguard the user's privacy.
There is currently no E2E encryption for iCloud, so short of HTTPS... what is this supposed to mean? There is literally no privacy at all if you upload to iCloud currently.
If anything, it feels like Apple's client side system enables something closer to E2EE while maintaining the ability to detect child porn on their platform.
>Apple's proposed technology works by continuously monitoring photos saved or shared on the user's iPhone, iPad, or Mac.
It does a check if it's being uploaded to iCloud Photos. It is not (currently, at least) continuously monitoring photos saved or shared; shared is Messages specific for child accounts.
>Because both checks are performed on the user's device, they have the potential to bypass any end-to-end encryption that would otherwise safeguard the user's privacy.
There is currently no E2E encryption for iCloud, so short of HTTPS... what is this supposed to mean? There is literally no privacy at all if you upload to iCloud currently.
If anything, it feels like Apple's client side system enables something closer to E2EE while maintaining the ability to detect child porn on their platform.
Unpopular opinion: I think the outrage over this is quite overblown.
This kind of hash checking is done by damn near all cloud providers - Google Photos, Dropbox, Gmail, Discord, Reddit, OneDrive, Facebook, Twitter, you name it. Apple have actually been very reluctant to implement this.
If you don’t like it, you don’t need to enable iCloud Photos. In the exact same way as if you don’t want your images scanned by Dropbox, you don’t upload the photos to Dropbox. It seems reasonable for Apple to implement something like to prevent iCloud becoming a repository for CSAM.
Edit: I’m unable to respond to the comment below this about hashes being performed locally, as I’m rate limited, but here’s my response anyway: The hashes are calculated on the local device, but only for photos which are about to be uploaded to iCloud Photos. The hashes are sent with the photos to iCloud. If a certain number of hashes in someone’s iCloud match (calculated using private set intersection), a manual review will be performed.
This kind of hash checking is done by damn near all cloud providers - Google Photos, Dropbox, Gmail, Discord, Reddit, OneDrive, Facebook, Twitter, you name it. Apple have actually been very reluctant to implement this.
If you don’t like it, you don’t need to enable iCloud Photos. In the exact same way as if you don’t want your images scanned by Dropbox, you don’t upload the photos to Dropbox. It seems reasonable for Apple to implement something like to prevent iCloud becoming a repository for CSAM.
Edit: I’m unable to respond to the comment below this about hashes being performed locally, as I’m rate limited, but here’s my response anyway: The hashes are calculated on the local device, but only for photos which are about to be uploaded to iCloud Photos. The hashes are sent with the photos to iCloud. If a certain number of hashes in someone’s iCloud match (calculated using private set intersection), a manual review will be performed.
Apple is forgetting the network effect of professionals that made them billionaires. I personally evangelized my clients in the past for years to switch to Apple ecosystem.
In my view this is weaponization of personal devices on a mass scale with clear intent of normalization of surveillance state on a global level.
The fact that this comes after NSO spyware investigation speaks volumes. They don't care about privacy. They care about control over user data.
I hope this will give power to Linux and FOSS movement and be the beginning of serious Apple detox.
In my view this is weaponization of personal devices on a mass scale with clear intent of normalization of surveillance state on a global level.
The fact that this comes after NSO spyware investigation speaks volumes. They don't care about privacy. They care about control over user data.
I hope this will give power to Linux and FOSS movement and be the beginning of serious Apple detox.
To introduce such invasive technology just a few weeks after the Pegasus scandal shows me that Apple has lost its ability to read the room.
https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
Apple uses sophisticated cryptography to make absolutely certain that you cannot hold them accountable for abuses of this system against you, NONE of which are prevented by its complex construction.
The private set intersection is an alternative to sending you a list of bad-image hashes which uses significantly more bandwidth than simply sending you the list. This alternative is superior for Apple because if they distributed the hashes it would be possible for someone to prove that they had begun matching against innocent images (such as ones connected to targeted races or religions, or sought to out particular pseudonyms by targeted images connected to them). It is inferior for the user for precisely the same reasons.
Some might be fooled into thinking the "threshold" behavior, somehow is in their interest: But no, Apple (or parties that have compromised them) can simply register the same images multiple times and bypass it and the privacy (for apple, but not for you) makes it impossible to detect that they've done that.
Apple uses sophisticated cryptography to make absolutely certain that you cannot hold them accountable for abuses of this system against you, NONE of which are prevented by its complex construction.
The private set intersection is an alternative to sending you a list of bad-image hashes which uses significantly more bandwidth than simply sending you the list. This alternative is superior for Apple because if they distributed the hashes it would be possible for someone to prove that they had begun matching against innocent images (such as ones connected to targeted races or religions, or sought to out particular pseudonyms by targeted images connected to them). It is inferior for the user for precisely the same reasons.
Some might be fooled into thinking the "threshold" behavior, somehow is in their interest: But no, Apple (or parties that have compromised them) can simply register the same images multiple times and bypass it and the privacy (for apple, but not for you) makes it impossible to detect that they've done that.
The common, instinctive reaction in tech circles & HN seems to be some version of "consumer action," vote-with-your-dollar stuff. This isn't going to work. It almost never does. At most, it'll be affect a minor tweak or delay.
This is a political issue, IMO. A thousand people protesting outside Apple HQ and/or stores is worth more than 100 thousand consumer striking. IMO, the main non-political avenue here is alternatives, especially FOSS alternatives. It's hard to see a way to widely adopted FOSS phones from here, but if we had a viable alternative things would look different. That's producer action, rather than consumer action.
Make it an issue in elections. Stand outside Apple stores with a sign. Push for a Digital Freedom law. Etc. An Apple labour union, maybe. The conscious consumer game is a dead end. It usually is.
This is a political issue, IMO. A thousand people protesting outside Apple HQ and/or stores is worth more than 100 thousand consumer striking. IMO, the main non-political avenue here is alternatives, especially FOSS alternatives. It's hard to see a way to widely adopted FOSS phones from here, but if we had a viable alternative things would look different. That's producer action, rather than consumer action.
Make it an issue in elections. Stand outside Apple stores with a sign. Push for a Digital Freedom law. Etc. An Apple labour union, maybe. The conscious consumer game is a dead end. It usually is.
[deleted]
But this made me wonder: is there an history of people complaining about this sort of things and actually achieving something?
I think this might have happened with MSFT's hailstorm/passport, where in the end industry opposition meant the project was abandoned, but I can't recall other instances.