Oliver Kuederle, who works with the image hashing technology used by Apple’s new technology that’s going to scan the photos on your iOS device continuously, explains that it is far, far from foolproof:
Perceptual hashes are messy. The simple fact that image data is reduced to a small number of bits leads to collisions and therefore false positives. When such algorithms are used to detect criminal activities, especially at Apple scale, many innocent people can potentially face serious problems.
My company’s customers are slightly inconvenienced by the failures of perceptual hashes (we have a UI in place that lets them make manual corrections). But when it comes to CSAM detection and its failure potential, that’s a whole different ball game. Needless to say, I’m quite worried about this.
This is just one of the many, many problems with what Apple announced yesterday.
Apple have stated that all suspect images will be validated by a human before passing onto authorities, so I don’t really see the problem here.
sloth,
Innocent people will have their privacy rights violated. Granted their privacy rights may not matter to you, but there’s no denying that this brings us closer to the 1984-esque world where people simply have no privacy expectations as it’s been relinquished to the authorities. It always starts with a “think of the children” argument, but there’s always an antiprivacy argument and it opens up the “if you’ve got nothing to hide, you’ve got nothing to fear” reasoning. Orwell’s works were prolific, however he thought the government would be the main culprit when it’s actually the corporations who are the ones making it happen. The irony isn’t lost on me that apple should be the company pushing towards this future given their own rebellious 1984 ad.
Mean while, Apple is taking a photo that is in your phone, flagging it, and then uploading it to a remote server, to be viewed and reviewed. How long is the photo going to stay, how many people are going to view it. What is the data retention policy….. there is just too much wrong…
Maybe if it was on icloud