What Google hides from us... for our own good?

Google's intention is to index the world's information. The assumption has always been that they would then share that with us.

Yesterday Google stopped reporting when a URL was in the main index or the supplemental index.

Perhaps Google stopped showing the supplemental label because webmasters and SEOrs freak out when one of their URLs are labeled as such.

Google withholding data so we don't freak out? There could be a trend for that.
  • Google could keep the PageRank toolbar data up to date - but chooses not to.
  • Google could keep the link: command up to date (or even accurate) - but chooses not to.
  • Google could keep the Google Directory up to date - but chooses not to.
The reason for this, according to an experimental post from Matt Cutts, is that Google noticed too many SEOs were suffering from "B.O." or backlink obsession.

It's great that Google publishes this data at all. It's all Google's data. They don't have to share it. However, it's slightly irking that I can't have the freshest data available.

Google also cracked down on synonym mining. Previously it had been possible to combine the minus (-) operator with the tilda (~) synonym operator in order to dig up "just synonyms" for words. For example, [-search ~search] would have listed (and highlighted) those words which Google saw as synonymous for "search". This technique does not work today and that's because Google didn't like SEOrs mining the data.

I suppose you could argue that Google once again took the "for your own good - we're withholding this data" approach.

One thing is for certain and that's the SEO community tends to get its knickers in a twist over loosing information like this. I'm sure dire warnings of the supplemental index were in the sales patter of a few SEO companies (again, perhaps it's a good thing Google's reduced its emphasis on the second index… perhaps not).

One of our Search Techies replied to the internal announcement about the death of the supplemental label with the following comment. And I agree.
The webmaster community seems to be freaking out about not being able to identify pages which are supplemental. Its fairly easy though if you look at a couple of cached timestamps. Takes a bit of monitoring but even still.

IMHO The supplemental index is one of Google's biggest failings. A below average response to make the best of a tough situation of crawling capacity.
3 comments

Popular Posts