The Peril of Dominance, or why we don't need a new Google

There was an interesting New Year's Day post on TechCrunch about something we've all noticed: for certain keyword phrases, Google is entirely spam. A search for a high-value keyword like "online degrees", for instance, turns up little more than affiliate directories run by spammers with a solid grasp of SEO. So many people are gaming Google that it has lost much of its value. But -- contrary to Wadhwa's implication -- this isn't a special failure on the part of Google's engineers. Rather, it's a fundamental characteristic of dominant technologies. Through market dominance, a technology can become the sole target of those who wish to exploit: an easy ROI for scammers, marketers and anyone else out to make a buck. Rather than building and optimizing for multiple competitive technologies, system gamers must only target one.

This is of particular concern for monopoly technologies, or borderline monopolies. Microsoft ran into the same problems fifteen years ago and continues to suffer the fallout. Hackers and virus creators knew that they only had to optimize for one operating system -- Windows -- and could target a massive share of the market. They ignored Unix and Mac OSes, giving those systems a reputation of relative security and safety against viruses and hackers. But have no doubt that if, say, Mac OS gained sufficient market share and corporate adoption, malware creators would see a new opportunity and begin writing viruses and malware for Macs. Suddenly, finding exploits in OS X would become orders of magnitude more important than it is today.

And thus Wadhwa's conculsion ("We need a new Google"), makes no sense. We don't need a new Google, an overwhelming search monopoly. We need a diversity of competitive search engines. Blekko's engineers are no better than Google's. And even if they were better, creating a search engine that is immune to gaming is fundamentally impossible, with increasing difficulty as the search engine's market share increases. Blekko is simply not spammed because it's not worth the spammers' time to figure it out.

Display advertising is a great counter-example of a market with diverse technologies, protocols and big players. While display isn't totally immune to gaming -- click arb and ads that launch malware, for instance -- it doesn't fundamentally challenge the value of the technology as overzealous SEO does to search.

When a technology is in a constant arms race with competitors, users win. When it is a black box inside a giant monopoly, the internet's underbelly rolls up its sleeves and gets to work.