Ask HN: Digital Ocean vs. AWS

[davismwfl]

We have been using AWS for the past 3 years, but looking at Digital Ocean simply because we could save some money if we moved over. So what are the plus/minuses of running node.js with mongo and RabbitMQ on DO compared to AWS? I am looking hopefully for people that have done this type of switch, either way, so I can learn from their mistakes. I am most concerned with uptime and reliability of the infrastructure.

For services we use, we mostly use EC2, Route53 and ELB's. I would think we would leave our Route53 as is, but move the EC2 instances and ELB workload. Is there an equivalent service at DO to ELB's or do I need to spin up instances and run our own load balancers? Don't care either way, just want to make sure we don't miss something and are comparing apples to apples.

[bkrull]

I've found DO is great for running RabbitMQ as a hub to process background jobs across multiple VMs. It's relatively simple to hit DO's API to spin up new "droplets" as needed so your queue never gets overloaded. I use this architecture to create over 1MM website thumbnails per day across 10+ VMs and it's been extremely reliable.

I currently use App Engine to handle front end requests and storage so I can't comment as much there, but DO's integration with Mesosphere sounds promising for scaling and reliability.

More here - https://digitalocean.mesosphere.com/ https://www.digitalocean.com/community/tutorials/an-introduc...

[shortoncash]

I'd be curious to learn more about the architecture you're using, as I have similar needs but in another space. How do I contact you?

[mattkrea]

DigitalOcean is just servers. While I use it for personal projects I would have a hard time moving a production workload that requires anything beyond a couple servers there.

While I love their service and recommend it often they would do well to add at least a few basic services (load balancers, S3-like storage, etc) on top of their current offering.

[davismwfl]

Thanks, from a cursory view that is what I saw, but figured I may be missing data.

At least to me DO seems like a great place to do a proof of concept and even maybe an early release, but we have real paying clients and while I'd like to reduce our costs, I don't want to introduce significant risk for our clients in terms of reliability or capability.

[shah_s]

They are both pretty great, and there are other companies such as linode and atlantic.net (disclaimer: i work for them) that might be good fit. AWS have a lot of services and features, but depends on what you are looking for. You can certainly run node.js, mongo, and rabbitmq with relative ease and cost on DO or anything. All the services I mentioned are pretty reliable so you don't need to worry about it. You can also use AWS for certain things, and others for your primary hosting. Anyways, good luck! Feel free to PM me if you have any specific questions regarding migration since we do it all the time.

[davismwfl]

Thanks, I may reach out. We talked with some hosting companies about co-locating too, so I am open to all ideas. Our core is we are growing and want to keep the reliability that AWS has given us, but see if there is a cost effective means to do so. Not that AWS has ever been bad to us, but we are bootstrapped and can't spend dollars if we can avoid it for now.

Question, atlantic.net as in the old Florida provider? If so you guys bought one of my businesses like 15 years ago. :)

[23david]

Moving from AWS to DO will force you to handle a lot more devops/IT stuff to work around network and api reliability issues. So to make that worth it, it'd really depend on how much money you are looking to save and on the IT/devops resources you have available.

AWS may be more expensive, but perhaps instead of migrating away from AWS to another provider, you can start using AWS more efficiently. Most people I've helped out in similar situations were able to reduce their AWS bills between 50-75%.

[davismwfl]

This is one of my concerns. We have spent years becoming AWS "experts" and while I want to save money, I also feel really comfortable with Amazon and their offers. We don't use a lot of custom solutions from AWS and are not heavily invested in devops peeps, so the dev staff serve both needs.

I can't say our bill from AWS is horrible, but I have challenged our team to reduce costs to try and make our process more cost effective so I can hire 1-2 more people. DO came up as an option. Not that we are spending that much with Amazon alone but that it may be an option to help save some money. To be totally honest, AWS has been so awesome to us it is hard for me to imagine leaving them, but I can't be blind to saving some cash. Thanks for the opinion and option.

[stephen-mw]

AWS is the best in the game and you pay a little extra for it. If you ask me I think it's worth it.

I wouldn't waste a single cycles migrating to another cloud providing to save a little money. In the end it simply won't be worth the time and you'll miss running on AWS's mature platform.

Instead, I would focus on getting your app working perfectly within linux containers. Use some existing tools such as swarm, CoreOS, or Tutum[0]. Once you have your app properly containerized, you'll be able to quickly move between clouds or even utilize multiple clouds (provided you have the time or experience to maintain the network).

The only way you'll save significant money is if you're willing to put up the resources to run your own prem, and I would strongly discourage you from doing that.

[0] https://www.tutum.co/

[nareshv]

If your app can run with just the h/w (CPU,Disk,RAM) available @ D-O you can do the migration. AFAIK, D-O does not have EBS like storage, which is essential in case there is lot of data that needs to be stored.

I have couple of sites (pretty low traffic) [nodejs, hhvm] running on D-O with pretty good uptime (> 1 year)

I'd recommend you to jot things down w.r.t what all things you have in terms of CPU, Disk, Memory, Apps, Server Usage over a week/month/year period and then see if the instances @ D-O can help.

Also, plan for future growth of the App, Just in case there is a sudden spike in traffic, there might be sudden need for additional Storage. In such scenario, with D-O you need to buy new servers, with EC2 you can add another disk with ease and can scale on storage.

AFAIK, D-O does not have ELB, so we need to buy instances and run ATS, Nginx, HAproxy, etc to do the load-balancing

[davismwfl]

Thanks.

We definitely are using ELB, S3, Route53 and the EC2 services. I am happy to hear that the uptime on your apps is really good, that makes me feel better, but given we use ELB's for our internal and external balancing it would be a consideration that we would have to run more servers and maintain that software. Not that ELB's are perfect, but so far they have been really reliable and "safe".

Thanks again for the input.

[lastofus]

I found DO's support lacking when my VM instance was down and unable to be brought back up (it eventually came back up on its own a few hours later, without any explanation for the downtime). It's something that has kept me from recommending DO for production use to companies.

[zacharybk]

Hey lastofus, I'm Zach, Director of Support at DigitalOcean. Your experience doesn't sound ideal and is certainly not common for us. Feel free to email me directly so I can look into your account and help make things right. My email is [email protected]. Thanks for raising this issue to my attention!

[davismwfl]

Yikes, that sounds totally unfriendly. In fairness so far you are the only one I have heard say that.

[kapilvt]

do make sure you secure internal inter-vm traffic with tls/ssl on digital ocean. there is no private networking, DO only offers shared 'private' networking that's common to all tenants. google compute engine is worth a look as well in terms of a 'real' cloud provider, cheaper prices & sustained usage billing discounts, instant scaling on load balancers, s3 api compatible file storage, etc.

[rubiquity]

I'm upvoting this so hopefully it gets more attention. You won't believe how many apps I've seen running on DO without TLS between their droplets. Setting TLS up right also is quite a bit of work when compared to Amazon's VPC which you can setup in just a few clicks.

[jdrumga]

AWS is a much more mature service. It is more of a work to create Load Balancers etc on DO as of now. Would highly recommend continuing with AWS for production environments. Our product is currently on DO but we would be migrating to AWS soon

[davismwfl]

That is what I was arguing but some devs that work for me are arguing we can save some cash and move to DO. I just don't want to dismiss any valid option until I understand the value and options. Thank you for the feedback.

[radnam]

You might want to check if digital ocean offers ELB or virtual IP as others might call it. Look at your detailed billing on AWS to help you make the decision. Another thing you might want to consider is how do you provision? How much effort would it take you to move from AMI to Ansible, Chef or such if you decided to move off AWS? You may also want to look at Linode (not affiliated with them)

[davismwfl]

Good point. Currently I have everything loaded in Ansible and we deploy using some custom scripts and Ansible to AWS. So I assumed, maybe poorly, that we could easily modify those to work with any provider. But you point is valid, we should investigate that before making any decisions.

Thanks for the input.

[kasia66]

Both of the cloud providers are great (AWS and DigitalOcean) - but all depends on what are you are going to use it for. DigitalOcean offers better value for money and is focusing more on developers who have smaller projects in hand, but AWS (although more pricey), will give you more additional features with the support for infrastructure.

If you are concerned about infrastructure why don't you check out www.cloud66.com. You might find it very helpful also you can have a look at the help page with the node.js migration to any cloud provider http://blog.cloud66.com/moving-your-nodejitsu-apps-to-cloud-.... (Disclaimer: I am working for Cloud 66)

[auganov]

I used to use DO for little side projects. It was good enough for that. There were random reboots but didn't mind it so much at the time.

When trying to pay (had some free credit before) they blocked my account and asked me to contact support to verify myself. The support person asked me if I can give them my fb or twitter account to verify myself. Kind of weird, but sure... I give them my fb, just to be told that my fb name doesn't match my CC name. Lol, I wouldn't give you my fb if I knew that's why you wanted it. And no, I cannot verify my CC name really, nobody has ever cared online in my whole life. I just gave up at this point and been using AWS ever since.

[api]

If you just need compute Digital Ocean, Vultr, Linode, and several others are all hands down better than EC2 for price/performance. I use Digital Ocean and Vultr and have had few issues with either of them in the reliability sphere either.

If you need permanent always-on systems (inelastic demand) I'd also look into OVH/Kimsufi, since they have extremely good deals on bare metal in their Quebec or French locations.

EC2 becomes attractive when you want to use Amazon's whole stack, or when you need the ability to flex up to enormous sizes. I'm not sure I'd bet on the smaller providers being able to hand you 10,000 instances on short notice.

[sandstrom]

# IPv6

One advantage with Digital Ocean is that they support IPv6[1] in 1+ region.

AWS still doesn't offer meaningful IPv6 support (not in VPCs, not on EC2 nodes, not for CloudFront).

[1] https://www.digitalocean.com/company/blog/announcing-ipv6-su...

# Cost

Can you give a rough breakdown of the bill itself, as well as utilization numbers (CPU) and number of instances over the last ~7 days.

That'll make it easier for people to provide feedback.

[infinii]

Back in the early 2000's, we were told (by media) we would run out of IPv4 addresses shortly and needed to move to IPv6. Over a decade later and we're still not there.

Who's actually using IPv6 now and why?

[breakingcups]

Additional question: Amazon instructs you to engineer for failure, and to expect any individual instance to fail at any moment.

How is DO in this regard? Do they randomly drop droplets or are they, for lack of a better word, persistent?

[europa]

If not using aws features like auto scaling , elastic load balancing etc. you most likely spending more money that needed if you are on AWS.

[edpichler]

If you want to save money in AWS, you can buy Reserved Instances. You will achieve relatively the same price of Digital Ocean.