Content Moderation for End-to-End Encrypted Messaging

Author(s): 
Publication Type: 
Other Writing
Publication Date: 
October 7, 2019

Thursday evening, the Attorney General, the Acting Homeland Security Secretary, and top law enforcement officials from the U.K. and Australia sent an open letter to Mark Zuckerberg. The letter emphasizes the scourge of child abuse content online, and the officials call on Facebook to press pause on end-to-end encryption for its messaging platforms.

The letter arrived the same week as a widely shared New York Times article, describing how reports of child abuse content are multiplying. (As the article notes, it is unclear how much of the increase is due to improved reporting and detection and how much is due to growing criminal activity.) The article provides a heartbreaking account of how the National Center for Missing and Exploited Children (NCMEC) and law enforcement agencies are overburdened and under-resourced in addressing horrible crimes against children.

The law enforcement letter, remarks at a Department of Justice event on Friday, the New York Times article, and new NCMEC materials on encryption appear to reflect a common technical assumption:

1. Content moderation is fundamentally incompatible with end-to-end encrypted messaging.

The law enforcement remarks appear to reflect an additional technical assumption:

2. Enabling content moderation for end-to-end encrypted messaging fundamentally poses the same challenges as enabling law enforcement access to message content.

My goal in this discussion paper is to provide a technical clarification for each of these points.

1. Forms of content moderation may be compatible with end-to-end encrypted messaging, without compromising important security principles or undermining policy values.

2. Enabling content moderation for end-to-end encrypted messaging is a different problem from enabling law enforcement access to message content. The problems involve different technical properties, different spaces of possible designs, and different information security and public policy implications. I aim to demonstrate these clarifications by formalizing specific content moderation properties for end-to-end encrypted messaging, then offering at least one possible protocol design for each property.

Read the full paper here