top | item 4939902

AWS: the good, the bad and the ugly

226 points| brkcmd | 13 years ago |blog.awe.sm | reply

85 comments

order
[+] blantonl|13 years ago|reply
This is a great writeup and is completely on target for realistic deployments on AWS.

We're a big user of AWS (well, relative, but we run about $10K/month in costs through AWS), so I'd like to supplement this outstanding blog post:

* I cannot emphasize enough how awesome Amazon's cost cuts are. It is really nice to wake up in the morning and see that 40% of your costs are now going to drop 20% next month going forward. (Like: Recent S3 cost cuts). In Louisiana, we call this Lagniappe (A little something extra.) We don't plan for it, nor budget for it, so it is a nice surprise every time it happens.

* We've also completely abandoned EBS in favor of ephemeral storage except in two places: some NFS and MySQL slaves that function as snapshot/backup hosts only.

* If the data you are storing isn't super critical, consider Amazon S3's reduced redundancy storage. When you approach the 30-50TB level, it makes a difference in costs.

* RDS is still just a dream for us, since we still don't have a comfort level with performance.

* Elasticache has definitely been a winner and allowed us to replace our dedicated memcache instances.

* We're doing some initial testing with Route 53 (Amazon's DNS services) and so far so good, with great flexibility and a nice API into DNS.

* We're scared to death of AWS SNS - we currently use SendGrid and a long trusted existing server for email delivery. Twillo will is our first choice for an upcoming SMS alerting project.

* If you are doing anything with streaming or high bandwidth work, AWS bandwidth is VERY expensive. We've opted to go with unmetered ports on a cluster of bare metal boxes with 1000TB.com. That easily saves us $1000's a month in bandwidth costs, and if we need overflow or have an outage there we can spin up temporary instances on AWS to provide short term coverage.

[+] ghshephard|13 years ago|reply
I don't get the 100tb.com model. Checking out their website, I see:

Intel E3-12303.2 GHz8GB2 x 1TB100 TB$201.15 1Gbit Dedicated Port

So, for $200/month, they'll give me 100 Terabytes/bandwidth on this server.

250 megabit/second at 30 days in terabytes = 81 Terabytes. A decently peered/connected Pipe costs, at this volume, around $5-$7/megabit @95th, or $1250.month.

So either:

  A) Their connectivity isn't hot.
  B) If you actually use that 100 Terabytes per server, 
     they start to curtail/rate limit/traffic shape you.
  C) Some other option I haven't considered?  Maybe they 
     just rely on the vast majority of their customers not 
     using that bandwidth, and subsidizing (significantly) 
     those that do?
blantonl, appreciate any insight you can provide - sounds like you've got great control over your environment. From the sounds of it, 100tb.com provides bandwidth @$0.33/megabit.
[+] vosper|13 years ago|reply
What do you see as the benefits of using Elasticache vs your own memcached servers? It seems to me that memcached is so simple to configure and manage that paying a premium to have someone else do it isn't that great of a win.

We switched to Elasticache recently as a test, and found that it works fine but don't see any compelling reason not to just manage our own memcached servers.

Edit: Also, what scares you about SNS? We use it for our oncall alerting and have seen no reason to be concerned about reliability - but I'd love to know if we should be!

[+] pearkes|13 years ago|reply
> We're scared to death of AWS SNS - we currently use SendGrid and a long trusted existing server for email delivery. Twillo will is our first choice for an upcoming SMS alerting project.

Can you elaborate on this a bit? Why are you scared of SNS? Data loss / latency etc?

[+] seldo|13 years ago|reply
My ops guy has reminded me that we do, like you, have one EBS drive for snapshots of one particularly hefty DB in addition to our other backup methods.
[+] falcolas|13 years ago|reply
> RDS is still just a dream for us

In my experience, with large data sets (I'd say > 100 GB), you want to run your own database instances, if only so you have control over the small fiddly bits you need control over.

RDS: 1) Uses drbd for master failover (this introduces network overhead on disk writes), 2) Only gives you access to super commands through stored procedures, and 3) doesn't let you tune InnoDB to your dataset or performance requirements.

[+] jaytaylor|13 years ago|reply
You might try using the SendHub API (https://www.sendhub.com) for your SMS alerts. Way easier to get started with than Twilio's, and the groups feature makes it very easy for people to subscribe to groups of alerts.

Disclaimer: I work at SendHub, but I've done it both ways and I definitely prefer ours ;)

[+] jbarham|13 years ago|reply
> We've opted to go with unmetered ports on a cluster of bare metal boxes with 1000TB.com.

FYI http://www.1000tb.com/ is the website for a landscaping company, not about bandwidth...

[+] trotsky|13 years ago|reply
The elasticity, no capex and api management are great. But bending over backwards to deal with their terrible SAN, saturated layer 2 and low reliability easily wastes at least as much engineer time as it saves in ops. If the OP's 100 boxes are similar to most of the other AWS deployments I'm familiar with it's likely that'd be 10 or less actual machines to manage.

Don't get me wrong - the idea is great and it's great for really low end workloads where the gotchas don't matter or really big systems where you'd be managing against most of those issues anyway.

But in the middle of the two there is a dead space that I bet a lot of shops are stuck in - double digit instance numbers running a workload that a 2-4 servers could handle with plenty of leg room.

If you're deciding on a cloud provider in 2012 I think it makes a lot of sense to shop around. There are lots of people doing the on demand api deployment thing now with different trade offs. I like joyent a lot (local reliable io) or providers with cloud and a colo area even if it's exorbitant - as paying five hundred dollars a month for instnaces one ssd could replace sucks.

[+] moe|13 years ago|reply
I second this. If you are in the position to commit either way I'd suggest to first run a simple benchmark with your app on EC2 versus some cheap dedicated server.

As parent says: EC2 does have its place, but for deployments in the 10-20 hosts range (or an app with "special needs") you tend to pay through the nose. Keep in mind that the amount that you pay for a fraction of a server on EC2 will rent you the entire box elsewhere.

[+] druiid|13 years ago|reply
Pretty much exactly this (I've posted this statement nearly word for word several times in response to articles like this). Basically EC2 or Rackspace Cloud, etc. are very good for small startups running a handful or two of instances, or really large companies that need hundreds/thousands of instances under which scenario hardware might very well be the more expensive option.
[+] leoc|13 years ago|reply
> The failure mode of EBS on Ubuntu is extremely severe: because EBS volumes are network drives masquerading as block devices http://joyent.com/blog/magical-block-store-when-abstractions... , they break abstractions in the Linux operating system. This has led to really terrible failure scenarios for us, where a failing EBS volume causes an entire box to lock up, leaving it inaccessible and affecting even operations that don’t have any direct requirement of disk activity.

Assuming that I/O is reliable and fast-ish was a bad OS design decision in the 1970s, as the Unix guys themelves soon realised, though it was an understandable mistake at the time. Continuing to develop and use OSes with an I/O interface that assumes I/O is reliable and fast-ish, in 2012 - it's pretty bad, isn't it?

[+] zurn|13 years ago|reply
Hardware and software have co-evolved, so that disks provide an illusion of error free operation until they throw in the towel and die. And they have consistent performance. This has worked OK so far.

With network filesystems (eg. NFS) you can choose to return an I/O error to the application when you hit a timeout or a network error (the -o intr mount option). This is rarely used since applications aren't used to dealing with them. So can't really blame the OS here either.

[+] dogas|13 years ago|reply
We here at PipelineDeals also abandoned EBS-backed instances after their 2nd outage.

Instead we rely on instances that use an instance-store root device. During the EBS outage, our instance store servers did not have any issues, while our EBS-backed servers really struggled throughout the day, with crazy high loads.

http://devblog.pipelinedeals.com/pipelinedeals-dev-blog/2012...

[+] ra|13 years ago|reply
Really interested to learn about how you use Chef to bring up and configure your ephemeral instances.

Also, do you backup your databases anywhere other than on other ephemeral instances?

[+] seldo|13 years ago|reply
Great post! It's fascinating to see that other people have come up with the exact some solution -- we also run a "skeleton crew" server setup in us-west.
[+] patrickgzill|13 years ago|reply
Have you done any calculations as to what it would cost to rent say, 20 x $100 a month dedicated servers spread across multiple datacenters, that can do virtualization with OpenVZ, Xen, or KVM (takes care of network, power, bandwidth, hardware issues) vs. what you spend monthly with AWS?

Bluntly it seems like you must have spent some dev or ops time learning all this and migrating away from EBS etc. even if you didn't hire someone.

Frankly, if your bills are greater than $3K per month with AWS I question whether you are truly saving anything.

(I figured that midsized instances vs. dedicated servers are about 5:1 in terms of performance)

[+] encoderer|13 years ago|reply
You can find essays from AWS users who swear it would be prohibitive--financially and mentally--to ever switch.

And you can find essays of people who run their own rack who swear the same thing about moving to AWS.

I worked at an all-AWS bay area startup that ran a $100k/mo Amazon bill, and now I work at a bay area startup that runs all its own hardware.

It's about tradeoffs. It's not just about cost. (Though there are many cases where Amazon is cheaper and it has little to do with your monthly spend, and certainly not an arbitrary threshold like $3k/mo.)

Writing this at the close of my employer's Open Enrollment period, I'd say this: Comparing AWS with running your own hardware is like trying to compare two health insurance plans. Each brings its own seemingly impossible tradeoffs. All you can do is bring your experience and judgment to identify core issues, make the best call you can, and suppress the urge to dream of the road not taken when you're knee deep in whatever hell you're experiencing that would be a non-issue if only you had went the other way.

Though if you loathe the AWS "fanboyism" so much then maybe it's the company you're keeping: Few currencies in a startup are as valuable as flexibility. The younger the startup, the truer that is.

[+] seldo|13 years ago|reply
If I knew how to setup OpenVZ and create < 10ms latency VPNs across multiple datacenters, then I would be a lot better at ops than I am. While I'm much better at ops now than I was when I co-founded awe.sm, I am still primarily a developer.

Our ops guy is much better than me, but his time is better spent working on higher-stack stuff like deployment automation, monitoring and efficiency tuning than on re-inventing a virtualization stack to save a few thousand dollars every month.

If we were bigger, it would be more worth the time and money spent. But without doing the math, we would have to be quite a lot bigger, I think.

[+] malachismith|13 years ago|reply
Thank you so much for sharing this. It's about time people started pointing out that EBS is a disaster. If we all do, maybe Amazon will finally fix it.
[+] jwilliams|13 years ago|reply
Nice read, but I wish they had included the location (and AZs) that are in use. I've used Oregon, California and Virginia with different results.

The comment around Ubuntu is interesting and I wish there was more detail there.

We use mdadm to run RAID across multiple EBSes. mdadm is great, but has a kink that it will boot to a recovery console if there the volume is "degraded" (i.e. any failure). This is even if the volume is still completely viable due to redundancy. This is obviously very bad, as you've got no way of accessing the console. It's an unfortunate way to completely hose an instance.

It's an easy one to miss, as you rarely test a boot process with a degraded volume. When it happens though - hurts a lot.

(If you'd like to check on this, make sure you have "BOOT_DEGRADED=yes" in /etc/initramfs-tools/conf.d/mdadm).

[+] seldo|13 years ago|reply
We are primarily in us-east-1, as I mentioned in the post, with a skeleton set of DB slaves sitting in us-west as an emergency recovery if all of east-1 goes down.

In terms of AZs, were are distributed roughly evenly across all AZs in east-1.

[+] IgorPartola|13 years ago|reply
> For these reasons, and our strong focus on uptime, we abandoned EBS entirely, starting about six months ago, at some considerable cost in operational complexity (mostly around how we do backups and restores). So far, it has been absolutely worth it in terms of observed external uptime.

So what do you use now for your persistent storage? This might be the most interesting part.

[+] seldo|13 years ago|reply
All our persistent storage is on "ephemeral" drives. If we lose the instance, we lose the data, so we have a lot of redundant slaves and backups (see my other comment).
[+] dirktheman|13 years ago|reply
Excellent post, and it reflects my experience with AWS, too. I really like the concept, and love S3 (and Glacier), EBS is just too unreliable for us to rely on for our entire business. Nice read, thanks!
[+] crcsmnky|13 years ago|reply
I'd be curious to hear about their backup/restore procedures with just ephemeral storage.
[+] seldo|13 years ago|reply
We use Percona's XtraDB streaming backup to take backups of our smaller databases, and incremental backups of the larger DBs. We store them in a series of places: on a backup instance within east-1, on a second dedicated backup instance in west-1 (in case east-1 ever bites the dust completely, such as during hurricane Sandy), and then long-term archival on S3. S3 is good for smaller databases but for our biggest ones it takes multiple hours to download a full archive, hence the backups in multiple places.

However, our primary strategy for uptime is redundancy -- every db has at least one slave, and we are spread across multiple availability zones.

[+] joevandyk|13 years ago|reply
With postgresql, you can do streaming replication to a few machines. I have one large machine on standby, and then a couple really small ones, some on a different provider that just receive the updated db data using pg_receivexlog.

Stuff not in a database goes on S3.

[+] dschiptsov|13 years ago|reply
Yes, ability to clone a new box you don't own in a minutes the only real advantage.

And for that very uncommon requirements you have to pay per Kb and per hour rates for having no service or guarantee at all.

And no, you still need a sysadmin who understand how AWS works and what to do when AWS says "Oops, your "_____" isn't available".)

[+] daemon13|13 years ago|reply
From alestic.com

>> Both EBS boot and instance-store AMI ids are listed, but I recommend you start with EBS boot AMIs.

Why two opposite recommendations from alestic.com [authority on AWS] and practitioners?

Not a flame - I am planning my AWS deploy strategy and need to make a decision between these two approaches.

[+] spartango|13 years ago|reply
As a bit of background: a lot of the performance pain around EBS is the inconsistency in performance between IOs. Many times you'll get a nice fast op, but sometimes IOs will get slow suddenly, or even stuck. You can imagine that internally, this is EBS seeking on magnetic disks or getting blocked by a clogged network. Regardless, inconsistent performance is nearly as bad (or perhaps worse) than consistently bad performance in certain cases:

Thus, most of the time, performance issues with EBS bite you when you have application data on an EBS volume. If you're constantly hitting the volume to serve data to customers, you'll be seeing the hiccups in EBS performance and passing them right on to customers. Clearly this is a suboptimal experience, and can lead to other failure modes; a slow/stuck bunch of ops can get an application to completely stall.

It's important to note here that EBS has no error reporting system or timeouts; common operating systems aren't very good at handling disk IO errors, so EBS will never produce them, even when its having issues. This lack of transparency can make handling/working around stuck EBS operations nigh impossible.

So, having experienced these pains, people generally say "Don't constantly use EBS if you care about your application performance/stability," and they aren't wrong.

That said, if you use EBS as a boot volume only, you are likely not hitting that volume enough to feel the pain of slow-IOs; once booted you won't touch it much. At the same time, using EBS as a boot volume has all kinds of conveniences: the ability to start and stop the instance without loosing data, snapshots and AMI creation, persistent machine-specific configuration, etc.

I hope that's useful. EBS is not a perfect service, but it definitely has its uses. EBS as a boot volume has made using AWS quite a bit friendlier.

Note: I worked on the EBS team at AWS a few years ago. I currently use EBS in my startup's infrastructure. If you have further questions, feel free to reach out to me.

[+] FireBeyond|13 years ago|reply
Good article - it discusses things in a good way to get an overall quick look into how the ecosystem works.

I'm yet to play with AWS on any significant level, but this is the kind of thing to bookmark.

Further to that, are there any recommendations for books/sites/entries that discuss more best practices?

[+] seldo|13 years ago|reply
I've not found any, which is part of why I wrote this post! It seems a lot of people blow time and money re-discovering these things.
[+] Shorel|13 years ago|reply
To me the decision is simple: if you need less than n servers, AWS is better and cheaper.

OTOH, if you need equal or more than n servers, rolling your own dedicated servers is better and cheaper.

The only issue is to determine the current value of n.

[+] pjungwir|13 years ago|reply
I've run a few projects on AWS and agree with how much it simplifies your life, but like the OP the big sticking point is EBS. I wasn't aware that ELB relies on EBS. Good to know!

I've also looked into ephemeral storage, but ultimately I decided to just rent a dedicated machine from elsewhere. Building a B2B site, I'm not as worried about massively scaling on a dime. The project has still used transient EC2 instances for a few odd things, though. It's nice to have that option when you need it!

[+] jiggy2011|13 years ago|reply
How much does AWS save in terms of admin vs a standard VPS?

I assume it runs a standard linux distro, therefor patches , firewalls, dependencies etc are still an issue surely?

[+] druiid|13 years ago|reply
Here is actually a real issue with many of these startup type companies using EC2 or similar services... one of the very benefits that are brought up (not needing to have someone building and maintaining servers) is also the person whom generally would be taking care of those services (or abstracting them away with CM solutions).

Many times what I've seen is simply that position is simply 'filled' by developers whom have ideas about designing and securing scalable systems, but when it comes right down to it... they get hacked within some period of time because it turns out, they don't know these things well enough to take the place of someone whom dedicates their time to it.

[+] ceejayoz|13 years ago|reply
> I assume it runs a standard linux distro, therefor patches , firewalls, dependencies etc are still an issue surely?

AWS's security groups pretty well cover you on the firewalls front (although some folks like to still run software ones on their instances to be doubly sure), but you're otherwise correct.

For a single instance, you're not going to see significant differences. It's running large clusters of instances you're going to see things being better than the average VPS provider, with stuff like Virtual Private Cloud etc.

[+] novaleaf|13 years ago|reply
from reading the "bad" parts in the article, man, what a pain in the butt. I guess i'm glad to be on Google app engine and not have to worry about this stuff.