Honestly, this is just saddening. Google needs to have a consistent policy towards these blackhat SEO offenders and enforce it for everyone. RapGenius bought backlinks to artificially inflate their pagerank. They knew they were cheating but they did it anyway. They got caught. And they were hit with a manual action on Google. Sounds good so far. But that's where it stops. RapGenius, using the connections they have from their multi-million dollars in VC funding, got back on Google and was able to avoid paying the price for their cheating.
Compare this to regular websites. Lots of smaller sites will end up paying a "whitehat SEO" firm to work on a "link strategy". These firms will claim up down and sideways that what they do is legal, ethical and follows Google's rules. But, what they actually do is either (1) create networks of fake sites to provide backlinks on certain terms to artificially boost the site, (2) place spam comments using bots on legitimate sites to do the same [not that this will thankfully no longer work well due to the latest Google algorithm update], or (3) pay legit sites to place backlinks to artificially transfer pagerank the same way that RapGenius did. Now, these other sites, when they get caught, they get a manual action or a smackdown. The difference? They have to actually pay the penalty. Arguing that they didn't know usually doesn't work. The penalty is LONG. They can't call on their VC firm to make calls at Google to give them a get out of jail free card.
I'd like to call on Google to create a public policy on how they handle these manual actions with some clearly defined penalties (example: 3 month manual action of 6 PR drop, etc) and to consistently enforce them across the board. That way a mom and pop site that pays a 'whitehat SEO firm' and gets caught doesn't have a worse time than a site like RapGenius that purposely engages in blackhat SEO who can use their VC connections to get out of having to pay a penalty in under 2 weeks. It's also sad that Google gave this get out of jail free card to a site whose entire business model is based around other people's copyrighted works which RapGenius doesn't have a license for, doesn't pay for, and publishes illegally.
As a search engine user, I don't care whether the search engine firm is "fair", whether their rules apply equally to everyone or not. It's not a country, it's a search engine.
You can say what you want about Rap Genius, but they're just about the only lyrics site that doesn't suck. As a searcher, I want to get to sites that don't suck. In the case of lyrics, that's Rap Genius.
Google has an interest to show me the best results. How they determine those results, and how much of that process is manual, is up to them. It's nice that they decided that penalties and virtual spankings are a good thing in the long run, and it's lovely that you think that they're too inconsistent and unfair about it, but I'm a user. I don't care. I just want the best results.
I keep hearing people bandying about this "VC connection" baseless innuendo. How is that anything but baseless, cynical speculation?
Remember, we're talking about someone (Matt Cutts) who penalized Google's own Chome website for improper links. If there was ever a conflict of interest, that would be it.
The idea that Matt was doing RG a special favor through a16z or other investors is an extraordinary claim, and we're going to need more evidence than just gut feeling.
They are back because they did a lot of work finding all these bad links and using Google’s disavowal tool to kill them, not because of their VC funding or whatever other influence you think they have on Google.
But if they post clearly defined limits and penalties, then that brings them into the territory of offering a "terms of service" for which they can potentially be sued.
So on many levels, keeping it vague and saying simply, in so many words "just be good, don't do anything abusive" like many services do for high-volume APIs (e.g. YouTube, last I checked) -- makes for a pretty good, common-sense fit in this case.
It also puts the emphasis were it should be (in my view): not on "don't do X or Y, OR ELSE" but on "look, just be ethical, and use your head, please."
It's just too bad for RG that they didn't head Google gentle and clearly (enough) worded advice, first time around.
Google is a business, and there is a very good chance that they financially benefit from Rap Genius being restored. Other companies that suffer don't have that financial link, and maybe the best course is for them to raise VC funds (and posture themselves so that Google stands to lose if they penalize)
I agree with you in theory, but in practice it sounds like you expect google to act as if they care about things like fairness. They are a company. The bottom line is what matters. If you think it's unfair, then try to hit the bottom line by using other services and convincing your friends and family to do the same
But this truly saddens me as well. I cannot for the life of me understand how they were able to fix ten-of-thousands of backlinks and other shady practices and get back in the good graces of Google so darn quickly.
Frankly I do not know a single other legitimate(+with good traffic) site who was able to get back in the good graces of Google after algorithm updates without spending many many months researching any possible reasons to be penalized and fixing it themselves after repeatedly asking google for help to no avail.
Could it be that Rap Genius was special because it has Andreessen Horowitz?
Sure seems that way.
I am not going to stop using Google, but I am inclined to use them lesser now.
I would like to hear why, how rap genius was fixed this and what this means for other people who get penalized from Matt Cutts though!!
They're back on Google, but keep in mind that they're seemingly not in a great position. Googling "justin beiber lyrics" doesn't show Rapgenius. Neither does "lady gaga lyrics" or "tupac lyrics". In all three cases, azlyrics.com has the top spot.
You have the causality wrong. Rap Genius has Andreessen Horowitz because Rap Genius is special.
In this case, the user experience for a Google user is worse when Rap Genius is removed. Google has some incentive to be more active in fixing the problem there, especially since Rap Genius was cooperative, technically competent, and aside from their "rapper swag persona", genuinely nice guys.
Having worked for large ftse100 publishers who had sites penalized in algo updates its stinks that such an avowedly black hat site gets of with a slap on the wrist.
> Frankly I do not know a single other legitimate(+with good traffic) site who was able to get back in the good graces of Google after algorithm updates without spending many many months researching any possible reasons to be penalized and fixing it themselves after repeatedly asking google for help to no avail.
Well there wasn't an algorithm update, there was a manual action. Google reversed the manual action after RapGenius stopped breaking the rules and cleaned up its mess. If this happened to you there would be a note about it in Webmaster Tools and you would be able to ask Google to take another look. I'm sure RapGenius had all hands on deck making sure they were up to snuff.
That they blatantly employed blackhat SEO techniques and are back on Google within a matter of days is sickening.
Think of all the sites that are penalized in various ways for similarly shady SEO practices and have to spend _months_ to regain their previous position.
Hopefully Bing et al can capture some marketshare and bring some degree of neutrality to the search engine landscape (i.e. not have every website owner on the planet beholden to the beast).
I do not understand why people are offended that they came back quickly AFTER doing a bit of penance.
Should their investors not have made calls? However, it should be noted that they did their part.
One of my favorite quotes is this: "To save a drowning man, he must first give you his hand"
If the Rap Genius guys were incompetent and did not do their part in removing the links so quickly (see main story for technical details) there would be little or nothing their investors would have been able to do about that.
BTW, it also helped that people really found Rap Genius useful. I'm sure many searched had rapgenius appended to it. So the lesson is, make your app so good that when Google delists you, it will make them look bad.
Thankfully?, there is also a precedence for other offenders to use. if you can detail you have atoned for your sins and Google insists your you must do a certain time, you know where to turn to.
BTW, this is Hackernews, and I look forward to the discussion of the (de)merits of how they scrapped and analyzed 177,000 links and not espousing of anger that they survived the punishment.
I'm sure Mahbod (my favorite) and co have learned that you do not go 'daaawging' when you are in a hole. I wish them success.
Because it is a double standard. There are thousands of sites which get blacklisted by Google. I'm sure all of them fix their errors really quickly once they realize they are on the black list. And none of them get back on Google nearly as quickly as rap genius did.
If Google applied these rules consistently with all websites, I would have no problem with it, but they don't. This breaks the myth that Silicon Valley is a meritocracy - as it shows that the connections you have are as important as raw merit, if not more
I have mentioned this before in this thread, I'll do it again. I AM really happy for rap genius.
I also view this post from RapGenius as a kind of a PSA.
They used the reasonable doubt argument, and won, they could have kept quiet about it. But, they let people know that others could appeal on reasonable doubt too.
Which is commendable!
Unfortunately, the underlying message some people miss when they feel this is about bashing RapGenius is that it is not.
The outrage is against Google. More specifically about 3-4 things:
1. Google has penalized sites for way lesser, and refused to reconsider, even when reasonable doubt was jarringly obvious (ex. some sites affected by panda/penguin algo updates targeted towards content/blog farms were penalized for as less as having similar posts type of intra-site linking as all content farms were using it)
2. Google will never apply this reasonable doubt argument for anyone in the future, unless they have friends like the investors at Andreessen Horowitz.
3. Google has given no indication on how they modified the algorithm for this special circumstance, the question that people are thinking is more in regards with if this was a team decision, or an individual's special favor.
4. It's a slap on the face for all white hat SEO professionals, webmasters and founders who have been burned by Google's no bullshit policy. Again its not about the policy existing, that's just a fact everyone has to work with. It's because now enforcement is clearly optional dependent on connections.
Just to hijack on one of your points: "it also helped that people really found Rap Genius useful."
This is painfully true, and I know it has been discussed in prior discussions. Rap Genius is without a doubt THE BEST lyrics site available, hands down. Even if the other sites weren't ad-ridden scammy buggy shitty websites, if they got their acts together and actually TRIED, Rap Genius would still be top dog.
I don't care for the attitudes or personalities of the people in charge, and I am absolutely disgusted that they felt the need to scam their Google rank, but I am willing to see that some infractions can be rectified through penance, and am happy to know that they will not be forced to shut down because of this fiasco.
> "To save a drowning man, he must first give you his hand"
is very wrong from anything I ever read about the topic. Drowning leads to involuntary reactions that cannot be controlled, like trying to climb on the person trying to save you, endangering them as well. That's why approaching and grabbing a drowning person from behind is suggested, and maybe even knocking them out if need be. (unless you can throw them a lifesaver or something else they can grab safely, of course)
BTW, it also helped that people really found Rap Genius useful.
Er, what? You suggesting that Google execs are benchmarking the productivity of google searches? There are a million other "substitutes"...hence the shady SEO...
God this pisses me off to no end. So my site is targeted by negative SEO. (Yeah, I know, I know, negative SEO doesn't exist. Fuck off.) So who do I call? I've been penalized for months and there is nothing I can do about it. Nobody can help. But these fuckers are able to make a call and recover from a SELF INFLICTED wound in less than a week. I am being targeted by crap that is COMPLETELY OUT OF MY CONTROL and it's tough luck little buddy. The double standard here is fucking sickening.
This scares the crap out of me. There are some really shady people in my industry and I'm sure they wouldn't think twice about launching a negative SEO campaign against me. I wish Google and Matt Cutts would address this for real.
This post is complete bullshit. Of course they have relations. Maybe they cleaned their backlinks, but still I cannot believe how they got back so fast in SERP.
Don't be so spiteful. It might very well be that they have relations, don't make it sound like that's a bad thing. Besides that they also seem to just have cleaned up their act very thoroughly.
And about Franz Kafka, I think he'd turn in his grave if he heard about how petty people like yourself criticize a website that tries to make literary criticism of his work digestible and enjoyable for the masses.
One thing I know is that RapGenius is good about making a big stink, being in the news, and then having things work out in their favor, even if they were the ones doing a dumb thing in the first place.
They freaked out at Heroku with decent reason because it was costing them money, but if they ran their own infrastructure, that would have been a total non issue and at the scale they are running, why ARE they using Heroku?
They did some incredibly dumb SEO things because they are greedy and are willing to cheat to win. That is on RapGenius, but once they got banned it is somehow Google's fault and they should be reincluded? I don't get it.
RapGenius is great at one thing - creating controversy and getting press for it. I guess that works for them, but it seems like a very selfish way to grow a business. Instead of focusing on building value themselves, they are willing to tear down their business partners in public to get what they want.
I'm not sure why this behavior hasn't put them in the same category as Zynga, Groupon, Swoopo, and all the other companies the tech industry loves to hate.
They freaked out at Heroku with decent reason because it was costing them money, but if they ran their own infrastructure, that would have been a total non issue and at the scale they are running, why ARE they using Heroku?
Are you arguing that what Heroku was doing wasn't wrong?
I'm not sure why this behavior hasn't put them in the same category as Zynga, Groupon, Swoopo, and all the other companies the tech industry loves to hate.
Comments on every single RapGenius post on HN have been filled with vitriol toward the founders, their attitude, and the culture they foster.
They have the word 'lyrics' at the end of the slug for that post and a number of others which don't have any relevance to lyrics. Surely that's still bad SEO practice?
The fact that all our text and song page URLs end in "-lyrics" is a relic of a time when we hosted almost exclusively lyrics.
Making the changes required to host all kinds of annotated texts has been a big project and making the URLs make sense is something we haven't gotten around to.
The reason it's not a totally trivial change is that both users and texts share the same top-level namespaces, and so right now we use the "-lyrics" suffix to differentiate song and text pages from user profile pages.
We want to change the suffix to "-annotated" for non-music pages and will hopefully get around to in the next few weeks. To our knowledge the existing situation doesn't help with SEO (no one is searching "bartleby the scrivener lyrics"), and it's confusing, so we 100% agree that the current situation is bad.
Very observant... I wonder what Google would say about this. I would almost find this more egregious than their original offense due to the posts having nothing to do with lyrics.
Its interesting that despite their founders outspokeness, the influx of heavy weight VC, tech press expose's on their offices, etc., RapGenius is really no more or no less than a standard, multi-multi-page SEO site, no different than the countless others just like it. No brand, no user affinity, no direct traffic to speak of. Just a bunch of drift net pages crawled by Google waiting to pick off unsuspecting search traffic. My favorite line of this post was the last: "Much love. iOS app dropping next week!" - unless you are searching that term in google, Im not sure who that is addressed to.
Pretty interesting decision to include a lengthy "technical digression" in their SEO apology:
<< Technical Digression: How to scrape 178k URLs in Ruby in less than 15 minutes - Ok, so you have 178k URLs in a postgres database table. You need to scrape and analyze all of them and write the analysis back to the database. Then, once everything’s done, generate a CSV from the scraped data. >>
So which hit should google show for a query like "the fox lyrics"? Of course, it should be a page that contains the lyrics, but which one? The most popular one? The one shown by google will be the most popular one.
From an IR perspective, search for lyrics is somewhat strange. I don't think it's easy to say if some hit is more relevant than the other. Sure, more popular sites may offer better UX (or simply less annoying ads), but at the end of the day it's a search for facts.
I wouldn't be surprised if google will serve those lyrics themselves some day. Freebase (and hence the google knowledge graph) already contains tons and tons of information on pratically any popular song (organized in a confusion way via recordings, canonical versions and whatnot) - artist, genre, album, release dates, length of a recording, etc. I don't think adding the lyrics as well is totally out of question. After all multible KBs are user curated and users provides lyrics.
Just like they serve "fact-hits" from their knowledge graph for queries like "movies by spielberg", google should then be able to serve the lyrics directly. While this is bad for lyrics sites, imho it is convenient for users and follows the same prinicple like "movies by spielberg" or "wife tom cruise"
This really is the thing that struck me. All this talk about companies being destroyed and jobs lost and hardly a mention of worry about the power Google yields.
We need more competition in this space. Sites need a search engine with real market share to compete against Google.
This really shows that even in Silicon Valley, your success is driven as much by the connections you have and the people you know, rather than pure talent and merit.
The "technical digression" with Github code on how to do a mass scrape is cool, though not surprising (er...how else would you accomplish such a big data task...though I'm glad I'm not the only one who uses Typhoeus, a great but underused gem).
But what's surprising to me is that a company that knows how to automate things and work with data...why the f--- were they manually soliciting bloggers to post links? That seems seriously inefficient and comically backwards...as if Sergey Brin spent his early Google days modifying SERPs by editing and uploading Excel spreadsheets to the server.
It seems seriously inefficient to me because at this point, Rap Genius has a lot of traffic and a lot of linkage already. So did the few backlinks, per day, that RapGenius got, through lame blogmasters who would participate in such a thing...really have an effect on their SEO rank, i.e. is it so easy to still game Google?
Or, was RG just doing it because they thought it would have a worthwhile effect (keep in mind that this is one of their co-founders who was individually emailing people), i.e. RG was being naively optimistic?
But now, Rap Genius co-founder Ilan Zechory tells us the site has returned to Google's search results. "It takes a couple days for Google to re-index everything, so search results are a little wonky right now, but we are officially reinstated," Zechory said in an email.
Somewhat ironically, Zechory served as a project manager at Google for two years prior to founding Rap Genius in 2009.
>> Wow that is troubling. So site that gets banned gets unbanned quickly and founder just happens to be ex google project manager
I've followed SEO on and off for years and I wouldn't have thought that their strategy of allowing people to generate and embed links back to the site was shady at all. I even recommended something similar a few years ago in good faith.
They're useful, relevant, and it just seems like good marketing. These things will spread organically and drive traffic even if you remove the SEO benefits.
Could you say that Scribd or Slideshare are doing something similar by allowing people to embed documents in their websites along with a no follow link back to their site?
Allowing people to create links to your website isn't shady.
Paying people to link back to your website on high-value keywords like 'justin beiber lyrics' from blog posts on other sites to artificially inflate your pagerank is 100% blackhat SEO. The payment in this case was a Twitter post about the link provider's website.
Actually, it's understandable that Google set this on High Priority...
#1: it's a high traffic website, that is a startup and the whole deal got a lot of attention of us (HN Readers) + Matt even saw this here. So it was good to respond fast (no choice)
#2: RapGenius is dependant on Google (although they try to let you see it as a social network, but it's not), they would have gone bankrupt ( i suppose) without Google.
#3: Without a quick response and/or measure to fix their shit on their end, investors wouldn't invest any more money in RapGenius and Google would probably hurt a lot of future investments (how is this startup dependent on Google, ...)
BMW once got a penalty from Google, they didn't got easily off. But their business wasn't dependent on Google.
Either way, there are both pro and cons to each action. But overall, i think they handled if fairly well.
[+] [-] JohnTHaller|12 years ago|reply
Compare this to regular websites. Lots of smaller sites will end up paying a "whitehat SEO" firm to work on a "link strategy". These firms will claim up down and sideways that what they do is legal, ethical and follows Google's rules. But, what they actually do is either (1) create networks of fake sites to provide backlinks on certain terms to artificially boost the site, (2) place spam comments using bots on legitimate sites to do the same [not that this will thankfully no longer work well due to the latest Google algorithm update], or (3) pay legit sites to place backlinks to artificially transfer pagerank the same way that RapGenius did. Now, these other sites, when they get caught, they get a manual action or a smackdown. The difference? They have to actually pay the penalty. Arguing that they didn't know usually doesn't work. The penalty is LONG. They can't call on their VC firm to make calls at Google to give them a get out of jail free card.
I'd like to call on Google to create a public policy on how they handle these manual actions with some clearly defined penalties (example: 3 month manual action of 6 PR drop, etc) and to consistently enforce them across the board. That way a mom and pop site that pays a 'whitehat SEO firm' and gets caught doesn't have a worse time than a site like RapGenius that purposely engages in blackhat SEO who can use their VC connections to get out of having to pay a penalty in under 2 weeks. It's also sad that Google gave this get out of jail free card to a site whose entire business model is based around other people's copyrighted works which RapGenius doesn't have a license for, doesn't pay for, and publishes illegally.
[+] [-] skrebbel|12 years ago|reply
As a search engine user, I don't care whether the search engine firm is "fair", whether their rules apply equally to everyone or not. It's not a country, it's a search engine.
You can say what you want about Rap Genius, but they're just about the only lyrics site that doesn't suck. As a searcher, I want to get to sites that don't suck. In the case of lyrics, that's Rap Genius.
Google has an interest to show me the best results. How they determine those results, and how much of that process is manual, is up to them. It's nice that they decided that penalties and virtual spankings are a good thing in the long run, and it's lovely that you think that they're too inconsistent and unfair about it, but I'm a user. I don't care. I just want the best results.
[+] [-] argonaut|12 years ago|reply
Remember, we're talking about someone (Matt Cutts) who penalized Google's own Chome website for improper links. If there was ever a conflict of interest, that would be it.
The idea that Matt was doing RG a special favor through a16z or other investors is an extraordinary claim, and we're going to need more evidence than just gut feeling.
[+] [-] unbeli|12 years ago|reply
They are back because they did a lot of work finding all these bad links and using Google’s disavowal tool to kill them, not because of their VC funding or whatever other influence you think they have on Google.
[+] [-] bonemachine|12 years ago|reply
So on many levels, keeping it vague and saying simply, in so many words "just be good, don't do anything abusive" like many services do for high-volume APIs (e.g. YouTube, last I checked) -- makes for a pretty good, common-sense fit in this case.
It also puts the emphasis were it should be (in my view): not on "don't do X or Y, OR ELSE" but on "look, just be ethical, and use your head, please."
It's just too bad for RG that they didn't head Google gentle and clearly (enough) worded advice, first time around.
[+] [-] sheetjs|12 years ago|reply
I agree with you in theory, but in practice it sounds like you expect google to act as if they care about things like fairness. They are a company. The bottom line is what matters. If you think it's unfair, then try to hit the bottom line by using other services and convincing your friends and family to do the same
[+] [-] bushido|12 years ago|reply
But this truly saddens me as well. I cannot for the life of me understand how they were able to fix ten-of-thousands of backlinks and other shady practices and get back in the good graces of Google so darn quickly.
Frankly I do not know a single other legitimate(+with good traffic) site who was able to get back in the good graces of Google after algorithm updates without spending many many months researching any possible reasons to be penalized and fixing it themselves after repeatedly asking google for help to no avail.
Could it be that Rap Genius was special because it has Andreessen Horowitz?
Sure seems that way.
I am not going to stop using Google, but I am inclined to use them lesser now.
I would like to hear why, how rap genius was fixed this and what this means for other people who get penalized from Matt Cutts though!!
[+] [-] sillysaurus2|12 years ago|reply
[+] [-] rdl|12 years ago|reply
In this case, the user experience for a Google user is worse when Rap Genius is removed. Google has some incentive to be more active in fixing the problem there, especially since Rap Genius was cooperative, technically competent, and aside from their "rapper swag persona", genuinely nice guys.
[+] [-] walshemj|12 years ago|reply
[+] [-] jonknee|12 years ago|reply
Well there wasn't an algorithm update, there was a manual action. Google reversed the manual action after RapGenius stopped breaking the rules and cleaned up its mess. If this happened to you there would be a note about it in Webmaster Tools and you would be able to ask Google to take another look. I'm sure RapGenius had all hands on deck making sure they were up to snuff.
[+] [-] unknown|12 years ago|reply
[deleted]
[+] [-] virtualwhys|12 years ago|reply
That they blatantly employed blackhat SEO techniques and are back on Google within a matter of days is sickening.
Think of all the sites that are penalized in various ways for similarly shady SEO practices and have to spend _months_ to regain their previous position.
Hopefully Bing et al can capture some marketshare and bring some degree of neutrality to the search engine landscape (i.e. not have every website owner on the planet beholden to the beast).
[+] [-] OoTheNigerian|12 years ago|reply
Should their investors not have made calls? However, it should be noted that they did their part.
One of my favorite quotes is this: "To save a drowning man, he must first give you his hand"
If the Rap Genius guys were incompetent and did not do their part in removing the links so quickly (see main story for technical details) there would be little or nothing their investors would have been able to do about that.
BTW, it also helped that people really found Rap Genius useful. I'm sure many searched had rapgenius appended to it. So the lesson is, make your app so good that when Google delists you, it will make them look bad.
Thankfully?, there is also a precedence for other offenders to use. if you can detail you have atoned for your sins and Google insists your you must do a certain time, you know where to turn to.
BTW, this is Hackernews, and I look forward to the discussion of the (de)merits of how they scrapped and analyzed 177,000 links and not espousing of anger that they survived the punishment.
I'm sure Mahbod (my favorite) and co have learned that you do not go 'daaawging' when you are in a hole. I wish them success.
[+] [-] grimlck|12 years ago|reply
If Google applied these rules consistently with all websites, I would have no problem with it, but they don't. This breaks the myth that Silicon Valley is a meritocracy - as it shows that the connections you have are as important as raw merit, if not more
[+] [-] bushido|12 years ago|reply
I also view this post from RapGenius as a kind of a PSA.
They used the reasonable doubt argument, and won, they could have kept quiet about it. But, they let people know that others could appeal on reasonable doubt too.
Which is commendable!
Unfortunately, the underlying message some people miss when they feel this is about bashing RapGenius is that it is not.
The outrage is against Google. More specifically about 3-4 things:
1. Google has penalized sites for way lesser, and refused to reconsider, even when reasonable doubt was jarringly obvious (ex. some sites affected by panda/penguin algo updates targeted towards content/blog farms were penalized for as less as having similar posts type of intra-site linking as all content farms were using it)
2. Google will never apply this reasonable doubt argument for anyone in the future, unless they have friends like the investors at Andreessen Horowitz.
3. Google has given no indication on how they modified the algorithm for this special circumstance, the question that people are thinking is more in regards with if this was a team decision, or an individual's special favor.
4. It's a slap on the face for all white hat SEO professionals, webmasters and founders who have been burned by Google's no bullshit policy. Again its not about the policy existing, that's just a fact everyone has to work with. It's because now enforcement is clearly optional dependent on connections.
[+] [-] holyjaw|12 years ago|reply
This is painfully true, and I know it has been discussed in prior discussions. Rap Genius is without a doubt THE BEST lyrics site available, hands down. Even if the other sites weren't ad-ridden scammy buggy shitty websites, if they got their acts together and actually TRIED, Rap Genius would still be top dog.
I don't care for the attitudes or personalities of the people in charge, and I am absolutely disgusted that they felt the need to scam their Google rank, but I am willing to see that some infractions can be rectified through penance, and am happy to know that they will not be forced to shut down because of this fiasco.
[+] [-] PavlovsCat|12 years ago|reply
> "To save a drowning man, he must first give you his hand"
is very wrong from anything I ever read about the topic. Drowning leads to involuntary reactions that cannot be controlled, like trying to climb on the person trying to save you, endangering them as well. That's why approaching and grabbing a drowning person from behind is suggested, and maybe even knocking them out if need be. (unless you can throw them a lifesaver or something else they can grab safely, of course)
[+] [-] 001sky|12 years ago|reply
Er, what? You suggesting that Google execs are benchmarking the productivity of google searches? There are a million other "substitutes"...hence the shady SEO...
[+] [-] sbreaking6|12 years ago|reply
This is rapgenius. If it had to cross the line why on something so materially insignificant.
[+] [-] friscobob|12 years ago|reply
[+] [-] MitziMoto|12 years ago|reply
[+] [-] argonaut|12 years ago|reply
[+] [-] jarnix|12 years ago|reply
Anyway, Franz Kafka would die again if he saw his text on this site (http://poetry.rapgenius.com/Franz-kafka-a-dream-lyrics), so let's just forget about this site and don't use it.
[+] [-] tinco|12 years ago|reply
And about Franz Kafka, I think he'd turn in his grave if he heard about how petty people like yourself criticize a website that tries to make literary criticism of his work digestible and enjoyable for the masses.
[+] [-] programminggeek|12 years ago|reply
They freaked out at Heroku with decent reason because it was costing them money, but if they ran their own infrastructure, that would have been a total non issue and at the scale they are running, why ARE they using Heroku?
They did some incredibly dumb SEO things because they are greedy and are willing to cheat to win. That is on RapGenius, but once they got banned it is somehow Google's fault and they should be reincluded? I don't get it.
RapGenius is great at one thing - creating controversy and getting press for it. I guess that works for them, but it seems like a very selfish way to grow a business. Instead of focusing on building value themselves, they are willing to tear down their business partners in public to get what they want.
I'm not sure why this behavior hasn't put them in the same category as Zynga, Groupon, Swoopo, and all the other companies the tech industry loves to hate.
[+] [-] jmduke|12 years ago|reply
Are you arguing that what Heroku was doing wasn't wrong?
I'm not sure why this behavior hasn't put them in the same category as Zynga, Groupon, Swoopo, and all the other companies the tech industry loves to hate.
Comments on every single RapGenius post on HN have been filled with vitriol toward the founders, their attitude, and the culture they foster.
[+] [-] LukeB_UK|12 years ago|reply
[+] [-] MattBearman|12 years ago|reply
More shady SEO practices from Rap Genius, disappointed but not surprised.
[+] [-] tomlemon|12 years ago|reply
The fact that all our text and song page URLs end in "-lyrics" is a relic of a time when we hosted almost exclusively lyrics.
Making the changes required to host all kinds of annotated texts has been a big project and making the URLs make sense is something we haven't gotten around to.
The reason it's not a totally trivial change is that both users and texts share the same top-level namespaces, and so right now we use the "-lyrics" suffix to differentiate song and text pages from user profile pages.
We want to change the suffix to "-annotated" for non-music pages and will hopefully get around to in the next few weeks. To our knowledge the existing situation doesn't help with SEO (no one is searching "bartleby the scrivener lyrics"), and it's confusing, so we 100% agree that the current situation is bad.
For more info on the "Worse is Better" philosophy that caused the current jank situation to persist for so long, see this talk: https://www.youtube.com/watch?v=X45YY97FmL4
[+] [-] jarnix|12 years ago|reply
http://poetry.rapgenius.com/Franz-kafka-le-nouvel-avocat-lyr...
[+] [-] knapp|12 years ago|reply
And some of those are not even poems:
http://poetry.rapgenius.com/Herman-melville-bartleby-the-scr...
This one is particularly weird / egregious:
http://poetry.rapgenius.com/Engineering-genius-understanding...
[+] [-] dmn757|12 years ago|reply
[+] [-] elwell|12 years ago|reply
[+] [-] McKittrick|12 years ago|reply
[+] [-] Soviet|12 years ago|reply
[+] [-] kanamekun|12 years ago|reply
<< Technical Digression: How to scrape 178k URLs in Ruby in less than 15 minutes - Ok, so you have 178k URLs in a postgres database table. You need to scrape and analyze all of them and write the analysis back to the database. Then, once everything’s done, generate a CSV from the scraped data. >>
[+] [-] bjoernbu|12 years ago|reply
From an IR perspective, search for lyrics is somewhat strange. I don't think it's easy to say if some hit is more relevant than the other. Sure, more popular sites may offer better UX (or simply less annoying ads), but at the end of the day it's a search for facts.
I wouldn't be surprised if google will serve those lyrics themselves some day. Freebase (and hence the google knowledge graph) already contains tons and tons of information on pratically any popular song (organized in a confusion way via recordings, canonical versions and whatnot) - artist, genre, album, release dates, length of a recording, etc. I don't think adding the lyrics as well is totally out of question. After all multible KBs are user curated and users provides lyrics.
Just like they serve "fact-hits" from their knowledge graph for queries like "movies by spielberg", google should then be able to serve the lyrics directly. While this is bad for lyrics sites, imho it is convenient for users and follows the same prinicple like "movies by spielberg" or "wife tom cruise"
[+] [-] tszming|12 years ago|reply
[+] [-] kenjackson|12 years ago|reply
We need more competition in this space. Sites need a search engine with real market share to compete against Google.
[+] [-] grimlck|12 years ago|reply
[+] [-] theodoracharl|12 years ago|reply
[deleted]
[+] [-] danso|12 years ago|reply
But what's surprising to me is that a company that knows how to automate things and work with data...why the f--- were they manually soliciting bloggers to post links? That seems seriously inefficient and comically backwards...as if Sergey Brin spent his early Google days modifying SERPs by editing and uploading Excel spreadsheets to the server.
It seems seriously inefficient to me because at this point, Rap Genius has a lot of traffic and a lot of linkage already. So did the few backlinks, per day, that RapGenius got, through lame blogmasters who would participate in such a thing...really have an effect on their SEO rank, i.e. is it so easy to still game Google?
Or, was RG just doing it because they thought it would have a worthwhile effect (keep in mind that this is one of their co-founders who was individually emailing people), i.e. RG was being naively optimistic?
[+] [-] kenshiro_o|12 years ago|reply
Moreover, this article feels pretty neutral/humble and techy, a far cry from the haughty attitude their founders usually display in interviews.
[+] [-] dhorowirtz|12 years ago|reply
But now, Rap Genius co-founder Ilan Zechory tells us the site has returned to Google's search results. "It takes a couple days for Google to re-index everything, so search results are a little wonky right now, but we are officially reinstated," Zechory said in an email.
Somewhat ironically, Zechory served as a project manager at Google for two years prior to founding Rap Genius in 2009.
>> Wow that is troubling. So site that gets banned gets unbanned quickly and founder just happens to be ex google project manager
[+] [-] benjaminwootton|12 years ago|reply
They're useful, relevant, and it just seems like good marketing. These things will spread organically and drive traffic even if you remove the SEO benefits.
Could you say that Scribd or Slideshare are doing something similar by allowing people to embed documents in their websites along with a no follow link back to their site?
[+] [-] JohnTHaller|12 years ago|reply
Paying people to link back to your website on high-value keywords like 'justin beiber lyrics' from blog posts on other sites to artificially inflate your pagerank is 100% blackhat SEO. The payment in this case was a Twitter post about the link provider's website.
[+] [-] thirsteh|12 years ago|reply
[+] [-] sbreaking6|12 years ago|reply
[+] [-] NicoJuicy|12 years ago|reply
#1: it's a high traffic website, that is a startup and the whole deal got a lot of attention of us (HN Readers) + Matt even saw this here. So it was good to respond fast (no choice)
#2: RapGenius is dependant on Google (although they try to let you see it as a social network, but it's not), they would have gone bankrupt ( i suppose) without Google.
#3: Without a quick response and/or measure to fix their shit on their end, investors wouldn't invest any more money in RapGenius and Google would probably hurt a lot of future investments (how is this startup dependent on Google, ...)
BMW once got a penalty from Google, they didn't got easily off. But their business wasn't dependent on Google.
Either way, there are both pro and cons to each action. But overall, i think they handled if fairly well.