"My primary worry about corporate information-gathering on individuals has always been about governments' ability to access the information the companies collect. This is especially a concern with facial recognition data, given the severe privacy ramifications of putting a pervasive surveillance system in law enforcement's hands. When tech companies say they won't sell their FR tech to law enforcement, that sounds nice, but it means nothing unless there are strict controls on "downstream" access by law enforcement. If IBM refuses to sell to a police department, but does sell to some middleman customer that then turns around and sells the FR tech to the police, that defeats the whole point.
We saw this "middleman" problem in 2016, when Twitter gave access to user data to a company called Geofeedia. Geofeedia sold a social media surveillance system to law enforcement, despite Twitter's rules against the use of user data for surveillance. Twitter cut off Geofeedia once this abuse was discovered, but it illustrates how companies can be blind to downstream uses that are nominally prohibited. Cambridge Analytica is another good example. Those instances show that the middleman issue is a hard nut to crack.
Any company vowing not to sell FR tech to governments needs to explain how they're going to prevent these issues from happening and, if they fail in prevention, how they think they could remediate the damage after the fact. If companies don't come up with satisfactory answers to these questions, well, maybe they need to revisit whether to work on FR tech at all."
- Date Published:06/18/2020
- Original Publication:Protocol