MySpace replaces all server hard disks with flash drives

[seiji]

The article is a blatant advertisement. Let me counter-advertise with my experiences here.

We did evaluations of a Fusion-io ioDrive card and a Texas Memory Systems RamSan card [1] recently. The ioDrive had strange performance characteristics (but nicely designed packaging). We ended up going with a few RamSan cards (though, they have much uglier packaging).

We have a few high throughput MySQL instances that needed to perform better. The database in question is only a few hundred gigabytes in size, so it's a perfect fit for the current generation of server flash cards.

Our path for improving improving performance went:

  - Started off with 32GB RAM and RAID-10 on SAS disks.
      - iowait sat between 10% and 15% constantly.
  - Moved up to 64GB RAM.
      - iowait cut in half.
  - Installed RamSan card and moved mysql with all databases to it.
      - iowait became negligible.

Now the server sits there with a few hundred gigabytes of flash, 64GB RAM, and it looks completely idle on usage graphs, but it's serving data faster than ever.

They are nice devices if you can afford them (and tolerate their quirks like needing to be completely formatted during firmware upgrades).

[1]: http://www.ramsan.com/products/ramsan-20.htm

[hristov]

Have you used them long enough to gauge longevity? How long do they last you? Do you have to use any special software to ensure there are not too many rewrites?

[angusdavis]

Curious, did you get these performance gains with off-the-shelf Mysql+myisam, or did you consider YC company rethinkdb (http://www.rethinkdb.com/)?

[c00p3r]

In many cases heavy raids is an overkill. just a hardware mirror + replication on slow backup servers could be a tradeof.

[timdorr]

This is the most interesting bit:

  MySpace's new servers also replaced its high-performance 
  hosts that held data in large RAM cache modules, a costly 
  method MySpace had been using in order to achieve the 
  necessary throughput to serve its relational databases. 
  MySpace said its new servers using the NAND flash memory 
  modules give it the same performance as its older RAM 
  servers.

Given Facebook's dependence on memcached (look at some of the work they've done at optimizing the Linux network stack) I wonder if this is something they're considering. This is a pretty big leap in terms of performance. I just wish the cost wasn't so insane.

And the longevity of these drives is a concern. What happens when you run out of good bits in the drive?

[j3fft]

IIRC, MySpace rolled their own "memcache" server... so that statement could be misleading.

[eru]

It's there some kind S.M.A.R.T. check with those SSDs like their is with hard drives, to give you a clue when the thing will fail?

[easp]

James Hamilton referenced this in a blog post today. He was pretty skeptical. He's been a proponent of SSDs in some applications, but he can't see how their cost can be justified yet on the basis of power efficiency alone.

The power efficiency + higher overall iops justify the up front cost in applications where there is a high ratio of iops/GB stored, but most data is not accessed frequently, and Im sure that social nets are no exception. It totally makes sense though to reduce the need for RAM caches, since it is both cheaper per GB than RAM, and draws less power.

[gojomo]

I'm surprised no virtual hosters are yet offering SSDs (as far as I know).

I suppose there's some risk one customer could burn out the SSD write-cycles then discard the node. Solution: charge for writes.

[lsb]

At the point when you'd need to vertically scale up from shared spinning platters to a SSD, you probably want the performance boost of running on the actual machine. ServerBeach has some Power Servers on sale that you can upgrade to a SSD for like $160/mo, roughly the cost of a reserved Medium instance.

[sadiq]

SoftLayer have Intel SSDs.

[dryicerx]

Conventional Drives or Solid State Drives, it's still the Internet's ghetto.

[mahmud]

Conventional Drives or Solid State Drives, it's still the Internet's ghetto.

Here is the racial implication of that statement :-)

http://www.cnn.com/2009/TECH/science/10/13/social.networking...