All evidence indicates that JC Penny was ranked #1 until the NYT provided Google with the evidence that it had collected.
Google then acted on the evidence. As a result of those actions, JC Penny's rank dropped from #1 to #71.
It is funny that he should say "Google's algorithms had started to work". Presumably, the algorithm doesn't depend on a New York Times reporter telling Google that they were being gamed :)
No, you are wrong about the timeline and what happened when. It is explained pretty clearly in the NYT article: "True, JCPenney.com’s showing in Google searches had declined slightly by Feb. 8, as the algorithm change began to take effect. In "comforter sets," Penney went from No. 1 to No. 7. In "sweater dresses," from No. 1 to No. 10." That happened the day before Google took manual action. BTW, going from #1 to #7 and #10 is not just a "slight decline."
At this point there's so much noise in Google Results that they're becoming an option of last resort. Not only is there a lot of noise, there's no real usable information from others on the quality of the results.
If I'm looking for a local vendor I go to Yelp, if I'm buying something I go to Amazon, and if I'm looking for coding help I go to Stack Overflow. I don't know that we necessarily need one site to replace Google; it seems to me that the era of free text search being the dominant paradigm may just be over.
I know what you mean about searching for local vendors or buying something; my experience is that a Google search isn't great for either of those. I still find Google pretty useful for coding help, though, and personally I don't tend to go straight to somewhere like Stack Overflow.
Could you give an example or two where Stack Overflow gets you an answer more quickly than a Google search?
Here's my example: In gdb, I want to list all of the threads in my running program. A Google search for "gdb list threads" gives me the answer in the first result. The same search on Stack Overflow doesn't seem to give me the answer in the first few results. This is hardly scientific, of course, so I'm interested to hear of coding questions where Stack Overflow does a better job than a Google search.
"the era of free text search being the dominant paradigm may just be over"
Perhaps it is the era of a global single instance search engine that is starting to come to an end - maybe we need a sensible way of having topic specific search engines and/or trusted content sites and allowing an easy way of treating these as if they were a global search engine.
Pretty much the same interface but a better implementation.
And yes, I know "meta" search engines have been around for a long time but, as far as I recall, don't try and identify the best places to perform your search - they just search run your search through a pile of generic search engines and aggregate the results.
This happened ~10 years ago when search-engine results were too messy for the throngs that were looking for the Dancing Baby, so the business world invented portals. Then portals became useless because their curatorial layers were not scalable, but by then search engines had made improvements and the pendulum swung the other way.
It's a natural see-saw between specialization and generalizations (there's probably a one-word term for this in evolutionary biology). We see the same in thin-client/thick-client talk in client-server/webapp contexts.
It's funny that someone who just sold out to AOL would choose to trash Demand Media.
> If Google was good at search, Demand Media wouldn’t exist.
Would Associated Content? Would AOL?
- - -
This is a cheap article that plays with words; it's titled "search still sucks", which is debatable but "truish", but what it really tries to say is that search "sucks more and more", which is provably false.
And it's 100% evidence free, too, as is the rule in the new entertaining noisosphere of blogism.
> So what is the evidence that search still sucks? Well, you know it’s true, just like me.
Mr. Arrington sounds like a priest now; what's the evidence that God exists? Well, you know it's true, don't you?
I was wondering why there were so few negative comments here on Michael Arrington posts.I have just read his biography on wikipedia.
Now I feel very naive. It Must be obvious I am new here. I understand no one here want to piss off this guy (like in my other comment). At least no one trying to launch a startup.
Search engines would benefit enormously from using data from social bookmarking sites. If you haven't lately used Delicious to search, I highly recommend it. The breadth of search isn't as far-reaching, but all of the results have essentially been pre-filtered by however many users have bookmarked it -- it's unlikely that many people would bookmark a spammy site or site with crap content.
I think Google results could vastly be improved if they tied together their normal rankings with a "how many people have bookmarked this URL (and to some extent, URLs from this domain)" metric. I've worked closely with Delicious' set of data, and it's nothing short of incredible. Billions of instances of people categorizing sites and vouching for their quality are going unused in the search space.
If black hat seos started to believe Google was paying specific attention to Delicious bookmarks, Delicious would be flooded with fake accounts faster than you can say "link farm".
History of search:
1. New company releases new search engine, better than the competition because it pays attention to a previously ignored signal.
2. Seos work out how to game the signal.
3. New search engine is now as crappy as the search engines it replaced.
Really, internet search is like macroeconomics: soon after you understand {how to prevent recessions, how to find the best webpages}, the problem vanishes and is replaced by something even more complex and incomprehensible. Furthermore, this has already happened.
The problem is that normal humans don't USE social bookmarking sites. Nor do they tweet/share much of value on facebook. Google is VERY good at serving up great results for things that you could imagine the "linkerati" might link to. Start drifting into travel, home remodeling, etc., and it gets pretty grim.
This is a type of sentience analysis. Google could also give better weights for links that are associated with things like "I love Ebay", rather than "[example.com] is a fraud!", but they don't yet (AFAIK).
Travel searches, for example, are a joke, and startups like Gogobot are popping up to try to fix that.
This is how Google is finally toppled. Not by a "google killer" or another giant like Bing, but by a thousand tiny niche services that get "X" search right.
This is true, but it has always been true. Google has always sucked for product review searches, local service vendors (try finding a plumber anyone?), travel searches etc. The odd thing is why people continue to use it for all these things. Perhaps the ads?
'Search still sucks' seems like quite an exaggeration when the average user (whoever that is) isn't even noticing any 'decline'.
I usually always find what I am looking for when using Google and haven't noticed any major changes recently when it comes to the search results. Maybe because I am taking some of my searches straight to respective sites like Wikipedia, IMDB, Discogs but still.
Also, ever since day one a bit of 'common sense' helps determining which results are actually useful (and I am all right with that).
Apparently, this blog author has been ousted as "bribed by Microsoft for secret advertising disguised in posts"
There has been a lot "google search sucks" submission from Techcrunch lately. This was starting to feel a little weird. So I googled "Michael Arrington Microsoft" and got this :
There's a lot of "google search sucks" lately because google search sucks. I would also be wary of techrights.org when it comes to regarding anything Microsoft.
Arrington was paid to appear in a Microsoft ad. Journalists usually don’t endorse companies or appear in their ads, especially if the also write about said companies.
That’s ethically questionable behavior but certainly not bribery. It has also no relation to the submitted article.
(I like how the author of the article you are linking to describes an interview with Ballmer as “spoke to Ballmer 4 months ago” which sounds much more suspect, especially considering the context. He certainly has a way with words.)
Heh that's Roy Schestowitz, I can remember him flooding Digg years ago. Nobody's certain if he's really a particularly hardcore anti-Microsoft troll or if he's actually paid by Microsoft because he's so ridiculously zealous it reflects poorly on everyone else.
Google loves what you guys call spam because it allows Google to generate more revenues.
When a content farm pushes first pages results this causes the real manufacturers and businesses to use Google's advertising platform to reach their customers.
When you're looking for a kitchen appliance It's better for Google to return a landing page filled with Google Adsense ads instead of the official website of one of these appliance manufactures. Again for the same reason mentioned above.
What this really means, is that Google should eliminate the middleman. Google shouldn't fight content farms. It should become the unstoppable juggernaut among content farms and do this by being actually useful and creating pages with value.
I'm afraid to say something like this, but some linkspam/scraper sites are verging on looking almost useful.
I was trying to Google references to Greek Playwright Aristophanes' use of fart jokes (yes, seriously I was looking for that) and I came across this linkspam page:
For one thing, this site doesn't offend my eyes. It doesn't immediately insult my intellect, either. After the first second, though, it still proves to be just as useless.
In a way, what Cuil was trying to do was to apply enough AI to make an actually very useful scraper site. What would the world be like if such sites actually existed?
I totally agree! Googles search results is starting to suck. If I'm sitting with an errorcode for some sort of program, trying to search it on google, will just give me a couple of random generated botsites.
I have moved permanently to Duckduckgo. :-)
I thought this was a great point: "When companies start to flail they nearly always do a couple of things. First, they trash the competitors. Then the talk about how hard the problem is and that the solution is a long term one."
This is why I think it'd be great to see Google release all the 40,000+ test queries which Google/Matt Cutts took back in 2000. It would help to put to rest some of the "Google is regressing and getting worse" debate one way or the other (and I suspect by showing that such talk is more sensationalist than accurate; I think Google search has improved a fair amount).
The article does make some interesting points, and I am a bit confused at the supposed sequence of events with the JC Penny/NY Times story. Plus it does raise the question of why a company who previously has had some SEO issues was - apparently - able to game Google's rankings in such a major way (using such spammy methods) without Google picking up on it for months.
That's a conclusion point from the article which I think is a well founded one.
As for the other points in the article's conclusion though:
>> "But all the evidence suggests otherwise. Demand Media is worth $1.6 billion, and their entire business is based on pushing cheap, useless content into Google to get a few stray links. If Google was good at search, Demand Media wouldn't exist."
"wouldn't exist" is subjective. If Demand Media weren't pumping out content, what would rank in its place? Part of Demand Media's success is that they are 'filling in the gaps' and covering content surrounding keywords which don't really warrant much competition. Sure this isn't always the case - and Demand Media's sites do sometimes rank highly for some rubbish content with other alternatives available - but in many cases it does seem to be at least part of the reason for DM's success.
Do I like some of Demand Media's content? No.
Do I like that Demand Media seem happy to have some pages with more adverts than content? No.
Do I think that Google et al should not even index some of their pages (some of which can at current rank fairly well)? Yes.
Do I think that there's many good alternatives to some of DM's content? NO. Unfortunately.
>> "And Bing wouldn't be making solid gains in search market share."
I might have different ideas about what "solid gains" means, although to me a percentage point here or there doesn't amount to much.
If - say - a Presidential candidate seen his/her opinion polls increase by 1 or 2%, you wouldn't see a bunch of stories saying how the candidate is making "solid gains".
In-fact, if a mainstream media source run an article on (say) a political party who seen a 2% rise in their opinion polls - and thus concluded that said party is making "solid gains" - I'd feel intellectually insulted and suspect the mainstream media source of bias.
So I definitely feel that this conclusion point is sensationalism (and inaccurate sensationalism, at that) and nothing more.
So all in all, a decent article which makes some decent points (at times). But it does also feel a bit of a shallow 'staying in the bandwagon' type of article, too.
If "the customer's always right", then, for the current search situation, you have to ask: Who's the customer?
--
P.S. Google: I'd accept, gratefully, as a stopgap, being able to specify multiple "site:foo" parameters. I don't know what the computational (or financial) impact to you would be.
(So far, I've held off trying browser extensions to accomplish this, but I guess I can start looking for one having a good reputation.)
Just tried DuckDuckGo on "windows autorun". What a world of difference. Top two links are recent news, and the third link is the authoritative KB from MS on disabling/re-enabling.
I tried DDG some months (years?) ago, but it was too inconsistent. Maybe it's ready for prime time, or at least the Late Late Show, now. Jazz hands!
[+] [-] credo|15 years ago|reply
All evidence indicates that JC Penny was ranked #1 until the NYT provided Google with the evidence that it had collected. Google then acted on the evidence. As a result of those actions, JC Penny's rank dropped from #1 to #71.
It is funny that he should say "Google's algorithms had started to work". Presumably, the algorithm doesn't depend on a New York Times reporter telling Google that they were being gamed :)
[+] [-] SoftwareMaven|15 years ago|reply
Algorithmic changes went into play 1.5 weeks ago and had started to drop JCPenny, then the spam team gave them some cement shoes.
[+] [-] zone411|15 years ago|reply
[+] [-] tomkarlo|15 years ago|reply
If I'm looking for a local vendor I go to Yelp, if I'm buying something I go to Amazon, and if I'm looking for coding help I go to Stack Overflow. I don't know that we necessarily need one site to replace Google; it seems to me that the era of free text search being the dominant paradigm may just be over.
[+] [-] chris_j|15 years ago|reply
Could you give an example or two where Stack Overflow gets you an answer more quickly than a Google search?
Here's my example: In gdb, I want to list all of the threads in my running program. A Google search for "gdb list threads" gives me the answer in the first result. The same search on Stack Overflow doesn't seem to give me the answer in the first few results. This is hardly scientific, of course, so I'm interested to hear of coding questions where Stack Overflow does a better job than a Google search.
[+] [-] arethuza|15 years ago|reply
Perhaps it is the era of a global single instance search engine that is starting to come to an end - maybe we need a sensible way of having topic specific search engines and/or trusted content sites and allowing an easy way of treating these as if they were a global search engine.
Pretty much the same interface but a better implementation.
And yes, I know "meta" search engines have been around for a long time but, as far as I recall, don't try and identify the best places to perform your search - they just search run your search through a pile of generic search engines and aggregate the results.
[+] [-] empire29|15 years ago|reply
[+] [-] rhizome|15 years ago|reply
It's a natural see-saw between specialization and generalizations (there's probably a one-word term for this in evolutionary biology). We see the same in thin-client/thick-client talk in client-server/webapp contexts.
[+] [-] bambax|15 years ago|reply
> If Google was good at search, Demand Media wouldn’t exist.
Would Associated Content? Would AOL?
- - -
This is a cheap article that plays with words; it's titled "search still sucks", which is debatable but "truish", but what it really tries to say is that search "sucks more and more", which is provably false.
And it's 100% evidence free, too, as is the rule in the new entertaining noisosphere of blogism.
> So what is the evidence that search still sucks? Well, you know it’s true, just like me.
Mr. Arrington sounds like a priest now; what's the evidence that God exists? Well, you know it's true, don't you?
-- Well, no, I don't. What else do you have?
-- Err, nothing.
[+] [-] zengeek|15 years ago|reply
Now I feel very naive. It Must be obvious I am new here. I understand no one here want to piss off this guy (like in my other comment). At least no one trying to launch a startup.
Ok now I need a new account.
[+] [-] photon_off|15 years ago|reply
I think Google results could vastly be improved if they tied together their normal rankings with a "how many people have bookmarked this URL (and to some extent, URLs from this domain)" metric. I've worked closely with Delicious' set of data, and it's nothing short of incredible. Billions of instances of people categorizing sites and vouching for their quality are going unused in the search space.
[+] [-] mooism2|15 years ago|reply
History of search:
1. New company releases new search engine, better than the competition because it pays attention to a previously ignored signal.
2. Seos work out how to game the signal.
3. New search engine is now as crappy as the search engines it replaced.
Really, internet search is like macroeconomics: soon after you understand {how to prevent recessions, how to find the best webpages}, the problem vanishes and is replaced by something even more complex and incomprehensible. Furthermore, this has already happened.
[+] [-] vanessafox|15 years ago|reply
If I recall correctly, Yahoo tested using this data a few years ago and found the signals not to be as useful as others they used at the time.
Of course, Bing and Google have been working to include more social signals in rankings: http://searchengineland.com/what-social-signals-do-google-bi...
[+] [-] webwright|15 years ago|reply
[+] [-] wisty|15 years ago|reply
[+] [-] noonespecial|15 years ago|reply
This is how Google is finally toppled. Not by a "google killer" or another giant like Bing, but by a thousand tiny niche services that get "X" search right.
[+] [-] pessimist|15 years ago|reply
[+] [-] jdp23|15 years ago|reply
[+] [-] soulclap|15 years ago|reply
I usually always find what I am looking for when using Google and haven't noticed any major changes recently when it comes to the search results. Maybe because I am taking some of my searches straight to respective sites like Wikipedia, IMDB, Discogs but still.
Also, ever since day one a bit of 'common sense' helps determining which results are actually useful (and I am all right with that).
[+] [-] zengeek|15 years ago|reply
There has been a lot "google search sucks" submission from Techcrunch lately. This was starting to feel a little weird. So I googled "Michael Arrington Microsoft" and got this :
http://techrights.org/2010/02/07/michael-arrington-microsoft...
Are people aware of this ? is it old news ? Is HN being spammed ?
[+] [-] itg|15 years ago|reply
[+] [-] ugh|15 years ago|reply
That’s ethically questionable behavior but certainly not bribery. It has also no relation to the submitted article.
(I like how the author of the article you are linking to describes an interview with Ballmer as “spoke to Ballmer 4 months ago” which sounds much more suspect, especially considering the context. He certainly has a way with words.)
[+] [-] benologist|15 years ago|reply
[+] [-] unknown|15 years ago|reply
[deleted]
[+] [-] rsardeha|15 years ago|reply
When a content farm pushes first pages results this causes the real manufacturers and businesses to use Google's advertising platform to reach their customers.
When you're looking for a kitchen appliance It's better for Google to return a landing page filled with Google Adsense ads instead of the official website of one of these appliance manufactures. Again for the same reason mentioned above.
It's all about the revenues.
- Reza Sardeha
[+] [-] stcredzero|15 years ago|reply
[+] [-] swingley|15 years ago|reply
[+] [-] stcredzero|15 years ago|reply
I was trying to Google references to Greek Playwright Aristophanes' use of fart jokes (yes, seriously I was looking for that) and I came across this linkspam page:
http://wn.com/2934_Aristophanes (seems to have dropped in Google ranking)
For one thing, this site doesn't offend my eyes. It doesn't immediately insult my intellect, either. After the first second, though, it still proves to be just as useless.
In a way, what Cuil was trying to do was to apply enough AI to make an actually very useful scraper site. What would the world be like if such sites actually existed?
http://wn.com/linkspam
(Seems like the site is just trawling through my cookies to make guesses about what I'd like.)
[+] [-] Strunk|15 years ago|reply
[+] [-] jdp23|15 years ago|reply
[+] [-] hammock|15 years ago|reply
Google /= search, it's just one engine. No need to throw all your eggs in one basket.
[+] [-] unknown|15 years ago|reply
[deleted]
[+] [-] tristanperry|15 years ago|reply
The article does make some interesting points, and I am a bit confused at the supposed sequence of events with the JC Penny/NY Times story. Plus it does raise the question of why a company who previously has had some SEO issues was - apparently - able to game Google's rankings in such a major way (using such spammy methods) without Google picking up on it for months.
That's a conclusion point from the article which I think is a well founded one.
As for the other points in the article's conclusion though:
>> "But all the evidence suggests otherwise. Demand Media is worth $1.6 billion, and their entire business is based on pushing cheap, useless content into Google to get a few stray links. If Google was good at search, Demand Media wouldn't exist."
"wouldn't exist" is subjective. If Demand Media weren't pumping out content, what would rank in its place? Part of Demand Media's success is that they are 'filling in the gaps' and covering content surrounding keywords which don't really warrant much competition. Sure this isn't always the case - and Demand Media's sites do sometimes rank highly for some rubbish content with other alternatives available - but in many cases it does seem to be at least part of the reason for DM's success.
Do I like some of Demand Media's content? No.
Do I like that Demand Media seem happy to have some pages with more adverts than content? No.
Do I think that Google et al should not even index some of their pages (some of which can at current rank fairly well)? Yes.
Do I think that there's many good alternatives to some of DM's content? NO. Unfortunately.
>> "And Bing wouldn't be making solid gains in search market share."
I might have different ideas about what "solid gains" means, although to me a percentage point here or there doesn't amount to much.
If - say - a Presidential candidate seen his/her opinion polls increase by 1 or 2%, you wouldn't see a bunch of stories saying how the candidate is making "solid gains".
In-fact, if a mainstream media source run an article on (say) a political party who seen a 2% rise in their opinion polls - and thus concluded that said party is making "solid gains" - I'd feel intellectually insulted and suspect the mainstream media source of bias.
So I definitely feel that this conclusion point is sensationalism (and inaccurate sensationalism, at that) and nothing more.
So all in all, a decent article which makes some decent points (at times). But it does also feel a bit of a shallow 'staying in the bandwagon' type of article, too.
[+] [-] pasbesoin|15 years ago|reply
--
P.S. Google: I'd accept, gratefully, as a stopgap, being able to specify multiple "site:foo" parameters. I don't know what the computational (or financial) impact to you would be.
(So far, I've held off trying browser extensions to accomplish this, but I guess I can start looking for one having a good reputation.)
[+] [-] vanessafox|15 years ago|reply
site:http://news.ycombinator.com OR site:stackoverflow.com <query>
I don't know if there's a max # of sites you can add. I tried it with three and it seemed to work great.
http://www.google.com/search?q=site:searchengineland.com+OR+...
(Yes, I did an ego search. Isn't that the best way to know if the results are likely accurate? :)
If there's a set of sites you want to search through often, you can set up a custom search engine for what you describe. For instance, see this page:
http://www.toddnemet.com/
[+] [-] pasbesoin|15 years ago|reply
I tried DDG some months (years?) ago, but it was too inconsistent. Maybe it's ready for prime time, or at least the Late Late Show, now. Jazz hands!