How Did This Get Published?

When you see a really silly editorial published in the New York Times you think... well, at least it's not the Post. But sometimes you really have to wonder about the agenda. By way of example, a guy who runs an insignificant company in England was given space to rant and rave about how his company is pretty much invisible to Google, speculate that it's because of a "penalty", and present absolute claptrap about how Google has no business advocating for network neutrality if his company can't outrank superior, vastly more popular websites.

Today, search engines like Google, Yahoo and Microsoft’s new Bing have become the Internet’s gatekeepers, and the crucial role they play in directing users to Web sites means they are now as essential a component of its infrastructure as the physical network itself. The F.C.C. needs to look beyond network neutrality and include “search neutrality”: the principle that search engines should have no editorial policies other than that their results be comprehensive, impartial and based solely on relevance.

The author's missive is directed at Google, not at Yahoo!, which seems odd given that Google is much more algorithm-driven than Yahoo!, is disinclined to "hand edit" even embarrassing search results (as compared to Yahoo!'s hand-editing, including to self-promote.

Moreover, in this quest for "neutrality", the author fails to specify what that concept means or how it could be measured. For example, one factor Google considers is whether people link to a site or its internal pages. If nobody's linking to the author's site, it's quite likely that the site isn't worthy of links - or that there are superior alternatives that get the links. Another big factor is unique content - sites like the author's, that rely almost exclusively on product feeds for their content, have virtually none. The author's site invites ratings, but I see no evidence that anybody has ever added a rating - theoretical unique content is not unique content. What do you get if you browse the site? Product listings, vendor information about the products, and affiliate links. Hardly a paradise for the consumer. And an overall site design and architecture that's not particularly search-engine friendly, relying heavily on iframes.

Reading the New York Times editorial you might be confused, and think that this is a big company that has invested in a serious innovation, yet cannot break through Google's iron wall. Hardly. This is a website founded by the author and his wife, with programming assistance from a family friend. It's the type of site a skilled programmer could knock off in an afternoon. It is no surprise to me that the site felt "punished" by Google, as a few years ago Google modified its algorithm to diminish the presence of sites just like the author's - sites that have essentially nothing to offer to the consumer beyond recycled information and affiliate links. Frankly, I personally think that too many site's like the author's still show up in Google's search results - and remain far too prevalent in Yahoo! Search and Bing. Unless and until it started to offer some real value, my ideal search engine wouldn't just relegate his site to the far reaches of the search results. My ideal search engine would exclude his site altogether.

Here's something I find odd. The author has been whining about problems with Google for years. The site was featured in a Guardian article several months ago. For all of the energy the author puts into complaining, it doesn't appear that he's expended any effort into improving his own website. Instead he whines in response to the suggestion that his site needs to present unique content that his site's replication of information easily found on other sources "is, in essence, all that Google itself does". Cute. Except Google tries to drive consumers to sites that match their search needs, rather than confining them to a set of affiliate merchants. And, unlike the author's company, Google does it well. And of course the author is begging the question - the issue is not that Google doesn't incorporate sites like his into the search results. The issue is that when there are tens, hundreds, thousands or tens of thousands of sites with the same content, those that offer nothing more than replication of the third party content deserve to rank at the bottom.

Further, the author claims to be in the same business as Google - the search business - and then complains that Google favors its own shopping search engine results over his. With all due respect, isn't that what you would expect a business competitor to do? But really, his site isn't a competitor with Google. People go to Google looking for information on products, and he hopes that they will come to his site from Google, follow an affiliate link, and make him a commission. Few people come to his site first, and none of them are directed to Google. From a search engine standpoint, he's an unnecessary middle step between the person searching for a product and the desired product - cut him out of the middle and the search engine user experience improves.

But don't just take my word for it. After the Guardian ran its story, there was a significant upward spike in traffic to the author's website. Even coming into the Christmas buying season, that spike didn't translate into any subsequent increase in traffic. That is, for the most part people seem to have taken a look and asked themselves, "That's all they have to offer?", then forgotten about the site.

The author also blames Google for a loss of traffic to MapQuest. I used MapQuest, often through its partnership with Yahoo!, quite regularly before Google Maps came out with its innovative AJAX interface and blew MapQuest's socks off. Google maps was a vastly superior product. Yahoo! doesn't even partner with MapQuest any more - is that Google's fault as well? Now it may hurt to be running a company and have a competitor produce a far superior product and take away your market share, but that's the way the markets are supposed to work. The same goes for the author's complaint that the share price of TomTom has dropped now that Google is making available a free turn-by-turn navigation service - what duty does Google owe TomTom's shareholders? Should companies be forbidden from giving away a service if a (sort-of) competitor would prefer to charge for a similar service? Should TV Guide be allowed to forbid cable companies from providing their customers with on-screen channel guides?

The author also whines that Google acquires technology from other companies. So what? Mergers, acquisitions, and the purchase and licensing of intellectual property play a big role in how companies operate.

Beyond that, the author's back to complaining that Google's cutting out the middleman - promoting its own search products through Universal Search instead of directing people to a third party website that it does not control in the hope that the third party will responsibly direct the consumer to an appropriate destination. Don't get me wrong here - Google's ability to leverage its way into new markets is a subject for valid concern - but it's not surprising that they favor their own sites when they're striving to provide a consistent, quality user experience.

So how did the story cross the pond, with no mention by the Times that the author is writing about a tiny, U.K.-based website? In my opinion, because the story was picked up by the industry groups that oppose network neutrality, and who hope that this type of idiocy can cloud the picture. We even get a silly parallel term, "search neutrality".

Google was quick to recognize the threat to openness and innovation posed by the market power of Internet service providers, and has long been a leading proponent of net neutrality. But it now faces a difficult choice. Will it embrace search neutrality as the logical extension to net neutrality that truly protects equal access to the Internet? Or will it try to argue that discriminatory market power is somehow dangerous in the hands of a cable or telecommunications company but harmless in the hands of an overwhelmingly dominant search engine?

There is absolutely no parallel between network neutrality and this nebulous concept of "search neutrality". None. You will not be able to get two people in a room to agree as to how a given set of websites should be comparatively ranked, let alone "all of them." Search engines have good cause to keep the details of their algorithms secret - it prevents people from gaming the system and spamming search results. For all the complaints about "how more transparency would be nice," this article from 2002 still does a pretty good job of what you need to do to make your site succeed. Sure, it's easier to take the author's approach and spin up a program that creates a website automatically from content created by others, but no secret here: it's rare at best for that approach to bring about long-term success.

More than that, in his zeal to attack Google, the author flips the idea of network neutrality on its head. Without network neutrality, the companies that offer Internet bandwidth can charge people at either end of a digital "transaction" for the privilege of sending their data over its wires - and without payment they could slow the transmission of the data down to a crawl or cut it off entirely. (Of course they could - and would - allow their own competing products across their networks, unimpeded.) You could pay more as a consumer for broader bandwidth, but perhaps still have your service provider narrow or cut off the bandwidth to sites or services you want to use. The service provider could also pay a fee for increased bandwidth.

Nobody's going to pay an extra penny to get access to the author's website, and he doesn't have the money to pay in their stead. Network neutrality will cost Google money, as it pays for the bandwidth - but they have the money with which to pay. And that will serve to cement their position (and that of companies like Microsoft) at the top of the heap, while a new, prospective competitor to Google - the tiny company that grows based on word of mouth, just as Google did as it quickly gained acclaim and displaced former search leader AltaVista, is unlikely to even have a chance.

So will the New York Times tell us, what lobbyist or telecom industry insider convinced them to run the editorial? (I'm not holding my breath.)