Tracking the Trackers: Early Results

Over the past several months researchers at the Stanford Security Lab have been developing a platform for measuring dynamic web content. One of our chief applications is a system for automated enforcement of Do Not Track by detecting the myriad forms of third-party tracking, including cookies, HTML5 storage, fingerprinting, and much more. While the software isn't quite polished enough for public release, we're eager to share some unexpected early results on the advertising ecosystem. Please bear in mind that these are preliminary findings from experimental software; our primary aims at this stage are developing the platform and validating the approach to third-party tracking detection. Many thanks to Jovanni Hernandez and Akshay Jagadeesh for their invaluable research assistance.


We began with a list of advertising companies that participate in the self-regulatory Network Advertising Initiative (NAI). By navigating popular websites we identified a piece of tracking content (primarily ads and beacons) from 64 of the 75 NAI member companies. We performed the following tests on each company's content:

1) Load the content.

2) Load the content, opt out of the company on the NAI website, and then reload the content.

3) Load the content, enable Do Not Track, and then reload the content.

We manually identified tracking cookies (cookies that appeared to contain a unique identifier or substantially unique information) and how they were altered throughout each test. A spreadsheet of results is available. Please email if you would like a copy of the data we logged while testing a particular company's content.

1. At least two NAI members are taking overt steps to respect Do Not Track.

Media6Degrees, an advertising data provider, deletes its tracking cookies and sets an opt-out cookie upon receiving a Do Not Track request.

BlueKai, a data provider and management platform, does not set tracking cookies in response to a Do Not Track request, but it does not delete any existing tracking cookies.

2. Over half Half of the NAI members we tested did not remove their tracking cookies after opting out.

NAI member companies pledge only to allow opting out of behavioral ad targeting, not tracking. Of the 64 companies we studied, 33 32 left tracking cookies in place after opting out.

3. At least eight NAI members promise to stop tracking after opting out, but nonetheless leave tracking cookies in place.

We compared our results to a survey of NAI member privacy and opt-out policies recently conducted by Carnegie Mellon's CyLab. We identified seven companies that (in the study's reading) promise to stop tracking when a user opts out, but nonetheless leave their tracking cookies in place.

The 24/7 Real Media privacy policy claims that a user may "opt out of receiving our ad delivery, audience management and behavioral targeting cookies." We found that opting out deleted the company's tracking cookies, but reloading the content reinstalled the tracking cookies.

Adconion's privacy policy states that a user is "free to opt out of the Adconion Cookie." Opting out deleted one of three tracking cookies but left the other two in place. Reloading the content did not update the remaining tracking cookies.

In its privacy policy, AudienceScience describes its opt-out option as follows: "Should you choose to opt-out, we delete all previously collected information from the cookies, and put new information in the cookie which tells us to stop collecting information from that device." We found that opting out of AudienceScience removes its unique tracking cookie but does not remove a highly unique cookie that represents the user's interests. Subsequent loads of the content updated the interest cookie.
[See below for an update from AudienceScience.]

Netmining's privacy policy states that upon opting out "we will delete your existing or cookie(s) and try to place a new cookie that instructs us not to track your future activities when we detect that cookie." Opting out deleted the Netmining tracking cookie but did not delete a tracking cookie served from a retailer-specific subdomain of (and presumably only used on that retailer's site). Reloading the content refreshed the retailer-specific cookie.

The Undertone privacy policy notifies users: "If you would like to opt out of OBA, then we offer 'opt-out cookies' to block the tracking and placement of future Undertone cookies for OBA purposes on your system for five (5) years." Opting out removed a highly unique cookie that stores the user's interests but did not remove a unique cookie. Subsequent loads of the content updated the unique cookie.

Vibrant Media's privacy policy provides: "If you'd like to opt-out from having Vibrant Media collect your Non-PII in connection with our Technology, please click here. When you opt out, we will place an opt-out cookie on your computer. The opt-out cookie tells us not to collect your Non-PII to tailor our online advertisement campaigns." Opting out of Vibrant Media does not remove the network's unique tracking cookie; the cookie remains in place and is updated with subsequent loads of the content.

The privacy policy on Wall Street on Demand's advertising platform claims: "By clicking here, the unique cookie used by this system/domain and stored locally by your browser will be changed to 'OPT_OUT'. By creating a generic cookie id instead of a unique cookie id - it is even more impossible to track your history." Opting out deleted Wall Street on Demand's unique cookie, but left in place a seemingly highly unique cookie that appears to store user interests. Refreshing the content renewed the interests cookie.

We identified one additional company with a privacy policy that may be interpreted to prohibit its current business practices. The TARGUSinfo AdAdvisor opt-out page explains that "[t]he AdAdvisor opt-out works by replacing the existing AdAdvisor cookie with a new cookie that clearly indicates that the user has elected to opt-out of the Services." Opting out left TARGUSinfo's unique tracking cookie in place. Refreshing the content did not update the tracking cookie.

4. At least ten NAI members go beyond their privacy policies and remove their tracking cookies.

In comparing our results to the Carnegie Mellon study of privacy policies we found that ten NAI members remove their tracking cookies upon opting out, even though they promise to only stop behavioral targeting of ads. The companies are: BlueKai (retains city-level geolocation), Dapper (bought by Yahoo!), FetchBack, Google, Invite Media, Media6Degrees, Mediaplex, Quantcast, TidalTV, and YuMe.

Concluding Thoughts

These early results scarcely scratch the surface of what we aim to learn with our new web measurement platform. We look forward to sharing new insights in the coming weeks and opening the software in the coming months. If you have experience in the web measurement field and would like to participate in testing the platform, please reach out. And please send web measurement questions — we're looking for new ways to put the system through its paces!


[If you would like us to add a statement from your company, please reach out.]

24/7 Real Media has updated its privacy policy.

You may also simply opt out of receiving interest-based advertising by clicking here.

AddThis contacted us about our findings. After a reevaluation, we discovered we had mislabeled a unique session cookie associated with AddThis's opt-out process as a tracking cookie. The post and spreadsheet have been updated. Our apologies to AddThis for the error.

AudienceScience reached out to clarify its practices. Its cookies store a compressed and encrypted data structure. When a user opts out, AudienceScience removes all interest segments and the unique ID from the data structure, but it continues to update the last time the browser contacted its servers. We have confirmed that AudienceScience now entirely removes its data structure after opting out.

BlueKai confirmed it is taking steps to honor Do Not Track.

Media6Degrees confirmed it is taking steps to honor Do Not Track.

Netmining has updated its privacy policy.

If you select the "opt out" button there for Netmining, we will delete your existing or online behavioral advertising cookie(s) and try to place a new cookie that instructs us not to track your future activities for the purposes of serving online behavioral advertising when we detect that cookie.

The Network Advertising Initiative has posted a response to the study.

TARGUSinfo submitted the following statement.

Immediately upon the publication of this study, we verified that our Opt-Out was fully functional both through our own site as well as through the NAI site. At no time was our opt-out not functioning, meaning that any consumer who had elected to opt out either through us or NAI or was indeed opted out, and no further activity was conducted on that user's browser. We did identify a minor inconsistency between the opt-out running on our own site and that which was running on the NAI site. Specifically, a second cookie was deleted when the opt-out was set from our own site, but that cookie was left on the browser if the user opted out through the NAI. Despite this cookie remaining on the browser, it was rendered dormant because our opt-out prevents us from reading or accessing any other cookie. We updated the code running on NAI to ensure that this second cookie also gets deleted when a user opts-out through NAI, to ensure that there is no confusion with our actual opt-out functionality and what was stated in our privacy policy.

Undertone has posted a statement responding to the study.

Vibrant Media submitted the following statement.

We drop a user ID cookie when a user initiates engagement with one of our ad units. This collects non-personally identifiable information on keywords a user has engaged with. If the user doesn't visit a site in our network for 10 days, we delete this data. If someone opts out, we add a do-not-track cookie.

We had been deleting any data associated with the user ID, but had not been deleting the cookie itself (this is acceptable for NAI compliance). When we encounter someone with a do-not-track cookie, we completely ignore the user ID and therefore don't use their information to serve ads. Although the cookie was remaining, we do not reference or use the ID in any way and we completely delete all data, be it in logs or storage devices for that particular user ID. Going forward, in order to prevent any misunderstanding we will also be deleting that cookie.

We have always been vigilant about adhering to industry best practices and NAI compliance policies.

Wall Street on Demand has updated its privacy policy.

Online Behavioral Advertising (OBA) is the process of targeting specific advertisements to each individual user, based on browsing history. If you opt out of OBA from our service by clicking the link below, the OBA cookie we use to contain this information will be emptied and changed to a placeholder signaling that you have done so. . . . Opting out does not necessarily delete or replace all cookies from our domain; others may remain which are used for aggregate reporting on the performance of the advertisements we serve.


Great research!
Have you tried checking whether the companies that violate their policies are certified by TRUSTe? (TRUSTe used to have a directory of all the companies they've certified, but no longer. Now you have to look up the company name in their search engine: If the company is TRUSTe-certified, you can file a Watchdog complaint with TRUSTe, and see if they correct their problems.

Great research!
Have you tried checking whether the companies that violate their policies are certified by TRUSTe? (TRUSTe used to have a directory of all the companies they've certified, but no longer. Now you have to look up the company name in their search engine: If the company is TRUSTe-certified, you can file a Watchdog complaint with TRUSTe, and see if they correct their problems.

Are cookies required to make sure that trackers know not to track? How are companies to know that users do not want to be tracked if there is not record on their system?

Close your browser and reopen after opting out?
Non-persistent cookies stay on your machine until you close the browser. If they change the cookie to be non-persistent after you opt out, by simply reloading the page, the cookie won't be removed. You need to kill all running instances of a given browser.
Multiple browser windows opened from the same browser (EG 3 chrome windows) will share cookies. Simply closing one browser or reloading a page won't get rid of it. Using the chrome example you need to kill all 3 or the cookie will stick around, even if it's been deleted.
Cookies with an expiration date (which are also stored as a file on disk) will actually be deleted.
You can browse all over the internet, and come back to the site and the cookie will still be there.
To test this, open gmail, but don't tell it to keep you logged in.
Log into gmail. Open a second browser window, close the first, browse to gmail. Your session will still be alive even though you told gmail to not keep you logged in, and you will already be logged in.
I believe your conclusions may be incorrect based on your published methodology.
As an aside, the only way to change this behavior is for browser companies to make their browsers work differently.
You may want to do some research and testing into how cookies work to improve your methodology or possibly go into more detail.
In addition you may want to simply ask them why the cookie is still there. It's quite possible, when you opt out, that they set a bit field in the database for your record to indicate that tracking data is not to be collected on your id, that you have been opted out.
I can think of very few other ways they could identify you as having opted out unless DoNotTrack (or some other authority) has servers available that get hit every time you are served an ad and look up your id with their own cookie.
This would require _immense_ server horsepower. There are hundreds of millions of ads served every minute on the internet. Each one would require that a request be sent to the authority's servers if every media provider was participating.
Knowing whether or not you are being tracked is a little more complicated than simply guessing based on the presence of a cookie.
As an HTTP expert (16 years working with the protocol at a low level, often directly through sockets) and programming professional, your results raised quite a few flags for me after reading your methodology.
I have worked in the ad industry. Indeed, in 1999 I wrote a third party ad server which audited media providers and provided ROI data on click-throughs to conversions.
I could tell where the ads were being served, when, if the user clicked them, what they bought, whether they filled out forms, pretty much the gamut of anything you'd want to track.

Our testing included cookie expiration time.
Do Not Track does not use a central authority. See

Guys- there's been a much simpler way that's been around for a while: change your browser setting to not accept cookies.
Simple, no?

You will be missing out on a great deal of web functionality in doing so.

All companies installed an opt-out cookie after opting out. We studied the other cookies that remained. To determine their purpose we looked at names, values, and in several close cases the responsible JavaScript. Feel free to reach out with any technical questions.

33 of the 64 companies left a cookie after the opt-out - my guess is that this is the opt-out cookie they set, so they'll be able to honor that opt-out request if they see that user again.
my company is one of the NAI members, if the authors of the study wish to contact me and discuss efforts to respect consumer privacy requests (cookie opt-outs, DNT requests, etc.).

Thanks for your work here.
I have always had problems with the idea that you actually need an "Opt-Out" Cookie from every tracking vendor to effectively "Opt-Out of receiving tracking Cookies from every tracking vendor. And then once you clear your browsers cookie Cache you have deleted the "Opt-Out" Cookies effectively Opting you into tracking once again.
This Opting-Out methodology is counter-intuitive and NOT likely to be understood by most internet users.
Enter Browser-based do-not-track headers. My hope would be pressure was applied to the online (and offline) tracking companies and browser vendors to support "Opt-Out" via DNT browser headers in addition to Cookie "Opt-Out".

Add new comment