Simple math, 618m/4000 is 154000 users per database (this is a horrible metric but illustrates my point) and 618m/9000 = 68666 and a bit users per memcache intances.
I hardly think this is a "fate worse than death", When you have a billion users with 600,000,000 DAU's whatever technology you use is going to have snags as you are pushing the envelope in just about every way.
This article reads like a Microsoft Whitepaper about why only a Microsoft technology running on a Microsoft platform will solve all your problems.
Stonebraker is known for posting ridiculous things like this in the SQL/NoSQL community. Look at the marketing hype for VoltDB for an idea. He also loves to rag on Riak, which is a pretty solid KV store. This is ridiculous
The sheer fact that FB was able to scale MySQL for their use till now proves that there is nothing wrong with MySQL. There are few apps of FB scale ever created. This would mean for most apps, MySQL is more than enough. This is another snake oil vendor peddling his unproven oil.
> "Anytime an engineer tells me that the only solution to something is to "bite the bullet and rewrite everything," I lose all respect for them"
Pretty closed minded approach. Just because it is usually not a good idea to do a rewrite, does not mean it is always true.
Here is an example which indicates you are probably wrong with that type of thinking:
What happens if a shop decides to store all their data in flat text files initially because it was easier and they were not expecting it to scale much. Later on and after huge expansion they realize that flat text files really is not the best solution, in fact it is pretty much the worst solution and decide that a rewrite with better technology is needed.
At this point you say you lose all respect for them? Does that attitude make sense?
Then it seems likely you're either correct and working with very poor engineers or making very poor and potentially incorrect assumptions based on your own lack of understanding of the code. I've been brought on to several projects over the years that simply could not move forward due to totally unsalvageable development failures; that's definitely not the case in every instance, but it absolutely does happen.
"Everything" in the context of the quote only refers (as far as I can tell) to code that accesses the database and it does not imply that the entirety of facebook needs to be transitioned simultaneously. It's a far cry from the naive "we'll rewrite the app and everything will be better" mentality that you seem to be referring to.
It's funny because Facebook is actually where Cassandra was born. Now under the Apache umbrella, it's odd that they didn't even mention it in that article. It's got by far the most momentum in the distributed database league.
"Facebook, for example, created Cassandra for certain tasks and also uses the Hadoop-based HBase heavily, but it’s still a MySQL shop for much of its core needs."
When a distinguished but elderly scientist states that something is possible,
he is almost certainly right.
When he states that something is impossible, he is very probably wrong.
The other two :
2.The only way of discovering the limits of the possible
is to venture a little way past them into the impossible.
3.Any sufficiently advanced technology is indistinguishable from magic.
... "running entirely in-memory instead of on disk." ...
Yeah one can see where problems might be. Why not just use the heap instead and have shards data + code and shard them that way? but then it won't be (R)DBMS would it.
[+] [-] noir_lord|12 years ago|reply
618 million Daily Active Users.
4000 shards, 9000 memcache instances.
Simple math, 618m/4000 is 154000 users per database (this is a horrible metric but illustrates my point) and 618m/9000 = 68666 and a bit users per memcache intances.
I hardly think this is a "fate worse than death", When you have a billion users with 600,000,000 DAU's whatever technology you use is going to have snags as you are pushing the envelope in just about every way.
This article reads like a Microsoft Whitepaper about why only a Microsoft technology running on a Microsoft platform will solve all your problems.
[+] [-] SEJeff|12 years ago|reply
[+] [-] pkolaczk|12 years ago|reply
Also, full ACID is overrated. Banks do well without it for years.
[+] [-] jbyers|12 years ago|reply
Previous comments: https://news.ycombinator.com/item?id=2740432
[+] [-] gregjor|12 years ago|reply
[+] [-] anuraj|12 years ago|reply
[+] [-] mrbill|12 years ago|reply
[+] [-] rgrieselhuber|12 years ago|reply
[+] [-] SeanDav|12 years ago|reply
Pretty closed minded approach. Just because it is usually not a good idea to do a rewrite, does not mean it is always true.
Here is an example which indicates you are probably wrong with that type of thinking:
What happens if a shop decides to store all their data in flat text files initially because it was easier and they were not expecting it to scale much. Later on and after huge expansion they realize that flat text files really is not the best solution, in fact it is pretty much the worst solution and decide that a rewrite with better technology is needed.
At this point you say you lose all respect for them? Does that attitude make sense?
[+] [-] obviouslygreen|12 years ago|reply
[+] [-] jjoonathan|12 years ago|reply
[+] [-] drsintoma|12 years ago|reply
[+] [-] AHconsidered|12 years ago|reply
[+] [-] leef|12 years ago|reply
[+] [-] cyunker|12 years ago|reply
[+] [-] eksith|12 years ago|reply
[+] [-] perlpimp|12 years ago|reply
[+] [-] ExpiredLink|12 years ago|reply