Microsoft is mad because Google is trying to finalize a deal that would give it a monopoly over the right to make digital copies of orphan books. But Microsoft is apparently in talks with News Corp. to obtain a monopoly over the right to make digital copies of News Corp's websites. News Corp. is mad because its content isn't making as much money as Rupert Murdoch wants it to. So his plan is to charge money for something nobody pays for -- the right to search and index websites. Murdoch hopes Microsoft is mad enough at Google to write a giant check for something everyone gets for free.
So where does all the madness lead? Nowhere.
The reason nobody pays for the right to index websites is because the law lets you do it for free. Yes, indexing typically involves making entire copies of websites, and yes that's something that might ordinarily create a copyright problem. But if you're making a copy of something for a good reason -- one that has a new and important purpose that adds value and functionality to the world -- and you're not undermining the underlying value of the thing you're copying, then copyright law will usually let you make that copy. It's called fair use. Here in the Ninth Circuit, we've thrashed out the fair use issue as it regards search engines, albeit primarily in the context of image search. The punchline is that copying content to facilitate search functionality is protected by fair use. That rule makes good sense. It creates immense social value by making it easier to sort through incomprehensibly vast quantities of information to find what you need in seconds.
Sometimes people get distracted, and think search engines can only crawl websites by permission, and that permission is provided (or withheld) by the robots.txt file. That's the file that tells search engines they can (or can't) index this site. Search engines have typically respected this "flag" but that doesn't mean they have to. But then again the fact they don't have to doesn't mean they won't. In fact, they'll probably continue to do so. That's because nobody is going to be crazy enough to start the doomsday machine Rupert Murdoch says he's ready to start.
Murdoch might be mad enough to tell Google it can't crawl his sites, even if that means a drastic loss in traffic. But he can only hope to recoup that loss if Microsoft is mad enough to pay for something it appears to have the right to get for free. Is that going to happen? I doubt it. Microsoft isn't so much mad at Google as it is jealous of its immensely profitable search business. But what happens to the search business if every website becomes convinced its entitled to charge every search engine for the right to crawl and index it? Well, lots of things. Most are bad for everybody. But they're particularly bad for the search engines. They will end up paying fees every time they crawl. (Or paying even bigger fees to lawyers to fight about this prospect.) And this will add up. Big time, site by site.
It's hard to imagine Microsoft will help destroy the profitability of the search business it covets. But even if Microsoft is mad enough at Google to take that road, where will it lead?
There might be a big fight. Google and other search engines might crawl anyway, on the strength of its fair use rights. The case would undoubtedly take years to sort out. In the meantime, Microsoft would wonder if it's paying truckloads of cash for something that's really free.
Or maybe there wouldn't be a big fight. Google and other search engines might respect the flag. Then what? Revolt, I imagine. Users will hate a system where they have to use one search engine to access one source of content, a difference search engine to access another source, and have to keep track of which is which along the way. And they will hate Microsoft for helping to create that system. News Corp. might not like the fallout, either. I can see all sorts of boycotts.
So Microsoft can start paying for things everybody gets for free, and take the risk of either paying a huge price that turn out to be free or alienating a huge swath of customers. Sweet deal. Is anyone mad enough to do that? I guess we'll see.