Google, Yahoo!, Microsoft, Twitter, Apple, Dropbox, LinkedIn, and Pinterest all publish transparency reports. Wordpress is the latest company to join the party, recently publishing their first transparency report. However, it’s difficult to see trends and anomalies when the information is scattered across multiple individual company reports. In order to get a comprehensive view of what is happening, we need to pull all of these fragments into a comprehensive picture. We need an internet industry-wide transparency report.
To create a kind of hacked industry transparency report, I have consolidated the July-December 2013 transparency data from the main internet companies. There is such a wealth of information to pore over and slice and dice in different ways that I will separate the analysis into a series of blog entries. My interest is the international aspect, so I will focus on requests from foreign law enforcement. This post will outline some of the key themes emerging from my comparison.
Only part of the picture
The first thing to note is that the transparency reports only show requests from foreign law enforcement that are made directly to the company. As I have previously noted, there are at least three main ways in which foreign law enforcement can access user data that is held by a US company:
1. through the US government via the mutual legal assistance treaty (MLAT) process;
2. directly asking the company; or
3. asking the FBI to obtain the data on their behalf.
The transparency reports only show requests made via method (2). This is not the companies’ fault; by the time an MLAT request filters down to companies’ inhouse lawyers and paralegals, it simply looks like a search warrant issued by the US District Court. For this reason, any requests that go through the MLAT system show up as US requests. This has the unfortunate consequence of over inflating the statistics for US government requests and hiding where the requests are really coming from.
Same-same but different
The companies’ reports are similar, but not the same. As the pioneer in this field, Google has set the template that many of the other companies now use. The Google-inspired template includes (1) number of requests (2) percentage of requests for which data was provided and (3) number of accounts/users affected. However, there are subtle differences in layout and content, which make it difficult to draw meaningful comparisons across companies.
For example, Microsoft divides their responses into percentages for content and non-content, which means that you can’t easily do a direct comparison with the percentages listed by companies that use the Google template. Of course, with some 8th grade algebra, you can overcome this, but it slows down the comparison process. A couple of other notable anomalies are that Twitter does not give a specific number for countries for which there were fewer than 10 requests. This could be because when the numbers of requests are small, criminal suspects may be tipped off that they are under suspicion. Dropbox does not break down their foreign requests by country at all (intriguingly, although 90 foreign requests were made, 0 accounts were affected – I can only imagine that this means that they refused all 90 requests).
There isn’t necessarily anything sinister going on here; it just makes direct comparison across companies difficult. It would be great to see consistent reporting across the internet companies (or even some consolidating reporting!).
Content vs non-content
One of the differences in the transparency reports that reveals a more substantive issue is that the Yahoo! and Microsoft reports indicate whether content or non-content has been handed over to foreign countries. This is more than just a question of template consistency; Yahoo! and Microsoft are notable exceptions in that they will accept jurisdiction for data requests in certain countries outside of the US. The other companies will generally only provide content through their US headquarters, in response to a US court order (eg through the MLAT process) or in emergency situations.
It’s harder to see overseas
There is an increase in the granularity of Google’s data with each year since 2010 and other companies are starting to follow suit. The improved transparency of national security requests is also a new development. However, when it comes to foreign requests, the level of detail takes a nosedive. There are at least two areas where this is important: legal process; and user notification.
Legal process
Google, LinkedIn and Dropbox break down the number of US requests according to which process was used eg subpoena, court order, or search warrant. This is not possible for foreign requests - as noted above, the only foreign law enforcement requests that are separately shown in the transparency reports are those that are made directly to the companies and this means that there has not been any US legal process. As I’ve noted [previously], the Electronic Communications Privacy Act does not apply to foreign governments, so they cannot use US legal process and companies have unfettered discretion about whether or not to provide non-content information to them. Some companies take this responsibility very seriously and apply their own due diligence processes before handing over user records. However, there is no visibility of what these standards and processes are. As I will discuss in a later post, there are notable differences between different companies’ compliance rates for requests from the same countries, which suggests that the companies may be exercising this discretion quite differently.
User notification
In the domestic context, Dropbox and Pinterest show whether they notified the user that their records had been accessed by US law enforcement. Apple, Google, Microsoft, and Facebook are all apparently updating their policies to increase their rate of user notification. Depending on the legal process used, ECPA establishes different obligations with respect to notification for access by law enforcement. This is quite an interesting statistic but it would arguably be more important to see the statistics for foreign requests. The fact that ECPA does not apply to foreign governments means that there is no obligation on companies to notify users if foreign law enforcement accesses their records. Since there is no legal obligation on companies, it is entirely a matter of policy as to whether or not they notify users and we have no visibility of how this plays out in practice.
I have attached the raw data from my unified industry transparency report, but will be sharing charts and diagrams that break up the data into more intelligible chunks over the next week or so.