top | item 6682324

Finding the right CDN for our startup

65 points| typerandom | 12 years ago |robin.userapp.io | reply

62 comments

order
[+] moot|12 years ago|reply
I've been extremely happy with CloudFlare. 4chan probably wouldn't be around today without them. CF has helped us by mitigating every DDoS since we started with them two years ago, and have kept operating costs in check by offering flat pricing. Even at their highest tier (Enterprise), we're saving money versus what we'd be spending with our colo.
[+] PLenz|12 years ago|reply
Another point for Cloudflare here. Seriously boost in site responsiveness after we went with them on our site. They also have a CDN of JS and CSS libs that we use extensively at http://cdnjs.com/
[+] opendomain|12 years ago|reply
I can not recommend CloudFlare enough. They offer a LOT of services for FREE! I have never had any problems and they use 2 factor auth.
[+] ommm|12 years ago|reply
It was due to 4chan's success with CF that I chose it for the projects I was the lead system architect for.
[+] tedivm|12 years ago|reply
I could write pages about managing CDNs, but unfortunately have to run. As an abbreviated version, Malwarebytes (company I work for) balances several CDNs around the globe to try to get the best performance possible (as well as to deal with some security issues inherent to hosting an antimalware tool). We have relationships with quite a few (not all of those relationships positive).

In my opinion this article skipped two of the best CDN's around, Edgecast and Highwinds. Edgecast in particular is damn fast (ssd boxes, high ram, most stuff served from memory). We're running multiple SSL'd domains through them without issue. Highwinds is more of an up and comer, but they have amazing service (as long as you avoid using them in South America).

The reason I avoid Cloudfront (AWS) is that it's way to expensive. I avoid CloudFlare because of the interactions I've had with their security team and CEO.

MaxCDN is good as long as you have no plans to go global. Let me be very explicit about this- if your CDN doesn't have a POP in Australia then they're not ready for prime time. There are only a few fiber lines going into the country and performance is absolute crap if you aren't hosting your files there. They also have nothing in South America, the MidEast or Africa.

As a seperate note, most DNS management companies will offer DNS based "CDN Managers", which will allow you to set priority rules for each CDN by region. This is an amazing tool that allows you to really take control of your traffic. I've never seen a CDN that was perfect everywhere, and when you have to deal with things like China (who don't really allow external CDN's in, forcing you to use one of theirs or host traffic out of Hong Kong) it's a life saver. I know that Edgecast, Dyn and UltraDNS offer these types of services (with the Edgecast one being the cheapest by far).

[+] jread|12 years ago|reply
I created a browser based test a few years ago that captures CDN latency from real users and correlates the results regionally using MaxMind (http://cloudharmony.com/speedtest). CDNs set themselves apart outside of the US where infrastructure is more complex and costs are higher. I posted a spreadsheet that provides a breakdown on CDN latency globally based on this testing (tens of thousands of tests from thousands of ISPs):

https://docs.google.com/spreadsheet/ccc?key=0Al_R0WNcU3q9dGV...

[+] neilk|12 years ago|reply
How can you do a fair test across so many different providers? Presumably you did not host the exact same content with every single service.

Also, I'd be interested in seeing the results for Fastly.

[+] cmer|12 years ago|reply
We switched from Cloudfront to Fastly and couldn't be happier. These guys are awesome and the product is top notch. I highly recommend Fastly.

We weren't too impressed with Cloudfront. The performance was all over the place and they provide no insight for cache hits/misses.

[+] typerandom|12 years ago|reply
I actually looked at Fastly just a few days ago. Unfortunately their custom SSL fees are a bit high. But seems really nice and will definitely do a write-up if we try them.
[+] twistedpair|12 years ago|reply
On price Fastly has copied every CloudFront price to the cent. I seems they differentiate on statistics and rapid purging (their purported competitive edge). However, for most any application, with proper version naming of resources, purges should not be an issue, unless you've got hard coded references to those resources.
[+] stevedomin|12 years ago|reply
I'd highly recommend Fastly too. Great product and great support. The guys there are doing an impressive job.
[+] peteretep|12 years ago|reply
I've not used Fastly, but I know how competent the guys behind it are, and I'd start any search there...
[+] wpeterson|12 years ago|reply
We had the same experience: CloudFront to Fastly.

Fastly has a great CDN service, beutiful origin hosting for a Rails or Python app, and customizable output filters (gzip? yes, please) and customizable cache config.

Can't recommend them enough.

[+] noelwelsh|12 years ago|reply
We use MaxCDN. It seems to work well, but I find all CDNs fairly opaque. How can I actually measure latency? I can't with web browsers being the way they are, so I really have no way of knowing. (There is http://www.w3.org/TR/navigation-timing/ but it is very limited, and not suitable for us.)

Clarification psated from a comment I made below:

Sure, I can test my latency, but I'm more interested in what my customers are experiencing. For instance, we have a big customer over in east Asia and I have very little idea what latency they get. When we were using Cloudfront a customer in Australia complained that they were seeing timeouts. It seems there was a bug in Amazon's routing table as Cloudfront has a POP in Sydney but their requests were going somewhere much further away.

[+] Erwin|12 years ago|reply
Google has this thing which lets you essentially choose a location (and browser) and run a test with output from the page speed addon: http://www.webpagetest.org/

I'm using Cloudfront with the origin option for a subset of our users. I've found a few clients with so restrictive or broken firewall rules that I had to add an option to default to getting the files from the source if users come from network X or Y. While cloudfront lets you download the log files, you get one S3 object per edge location per 15 minutes or something like that, so 1000s per day, which made things difficult to troubleshoot (I suspect the problem with one user was partial cache due to Apache's deflate sending responses as chunked encoding as default and Amazon's origin spider sometime dropping such connections).

[+] typerandom|12 years ago|reply
Pingdom's response time report is really nice. We also used LoadImpact to test it against "flash mobs" :)
[+] nwh|12 years ago|reply
All major browsers allow you to profile each load time of every single resource. Perfect for trying to compare relative latency at a particular location for a particular CDN. Chrome'a is especially good for this sort of thing, look at "network" in the developer tools.
[+] ksec|12 years ago|reply
The major problem is SSL Pricing. Which i have never looked at.

But purely in terms of speed, Akamai Rules. Closely Followed by ( and sometimes even exceed ) EdgeCast.

From my experience, MaxCDN is the next bet since they are from NetDNA. Not saying CloudFront, or Other OnApp based CDN not good, for large files transfer or video streaming they are perfectly fine. But for small files, fast response and low latency those three are the way to go.

Haven't tested Fastly yet purely because i think they are very expensive against some other established players. And I guess they have far few PoPs. Would love to see some detailed comparison though.

[+] tow21|12 years ago|reply
Also worth looking at OVH: http://www.ovh.co.uk/cdn/

£8.99/month including SSL.

(Note, I haven't actually tried it myself!)

[+] latch|12 years ago|reply
I briefly tested OVH's CDN, it wasn't great.

First, you use rules (patterns) to set the TTL, rather than headers. Second, I just couldn't get the SSL working. I'd upload the certificate, it would say that it would show up shortly, but days later, nothing. Third, the routing seemed messed[1]. Their support normally isn't horrible, but it took weeks to get anyone to acknowledge the support request that I opened on this, even after escalating it.

[1] I know people like to point out that internet routing has nothing to do with geography. But I squarely refuse to believe that, Singapore users should get routed to France (they have a POP in Singapore, HK and Japan).

[+] philip1209|12 years ago|reply
I do not understand why a custom domain with a custom SSL certificate was necessary. Why not serve SSL off of the Cloudfront domain at normal cost?
[+] jrnkntl|12 years ago|reply
I am not sure what the added benefit is (apart from the looks of using your own domain) when using your own custom SSL? Isn't the standard https connection through Cloudfronts wildcard SSL https://somethingsomething.cloudfront.net enough to be on the safe side? Really curious.
[+] noelwelsh|12 years ago|reply
Depends on the situation.

Our product (Myna; mynaweb.com) provides a JS client that our customers embed on their web sites. If we give them a Cloudfront URL we have just created a big legacy problem for ourselves if we ever want to move off Cloudfront (and we did and we have!)

If you're creating a pure JS client connecting to an API on a different domain you have to worry about CORS support to get cross site requests. It's not a huge problem but it slows your site down a bit (you have to make two requests to check CORS permissions and then send data, where you could make one without this issue) and you can't support legacy browsers.

So there are a few reasons why using a Cloudfront URL might not be 100% suitable.

[+] latch|12 years ago|reply
DNS provides an abstraction. If you use your own domain, you can change the underlying provider without breaking existing links.
[+] SudoAlex|12 years ago|reply
This is probably setting the requirements a bit too high for the average site - requiring a unique SSL certificate from their CDN provider.

Most sites will probably be fine using SSL for their main site using their main servers - requests going direct to the site, with static/media files being served from a shared SSL certificate with their CDN.

[+] anderspetersson|12 years ago|reply
I made the switch from Rackspace Cloudfiles + CDN to Amazon S3 + Cloudfront a couple months ago. I was regularly getting emails about my app erroring when someone uploaded a file via my app to Rackspace, since Rackspace was down (or responding slowly).

Now I'm happy with Amazon S3 + Cloudfront, it's cheaper and more stable.

[+] typerandom|12 years ago|reply
I wish we could use CloudFront. Unfortunately they charge a bit for custom SSL ($600/zone/mo). Not that fun for a startup. Are you also using that? /R
[+] lixef|12 years ago|reply
Since it's not all about the pricing for SSL pricing, I can recommend this site:

http://www.cdnplanet.com/

They provide a pretty nice overview about edge locations and other important options for considering to use a certain service.

[+] kalleboo|12 years ago|reply
Are the high SSL costs because they need a dedicated IP per domain at each of their POPs?

Would it be possible for them to offer an SNI-only SSL option for far cheaper if you know beforehand who your clients are (say, if you're hosting content for an iOS app)?

[+] twistedpair|12 years ago|reply
CloudFront is the same, $600/mo/cert. They claim it is not so much the cert as the need for a dedicated IPv4 address at each edge node (45 edges and counting), and the paucity of addresses left in said pool.

CloudFront has been asking a lot about a hypothetical SNI only offering in their user surveys lately, so that is likely the route they'll pursue shortly for far cheaper SSL options.

[+] stellar678|12 years ago|reply
CDN.net offers the Subject Alternate Name option for $199/year. I've been doing research but we haven't tried them yet.

Edit: Just recalled they use Subject Alternate Name, not SNI. This is where they add your domain to their SSL certificate.

[+] slowdown|12 years ago|reply
Prospective Rackspace customers - Avoid the cloudfiles offering from Rackspace at all costs. Their support is good, but we used to average out atleast one user who were getting access denied to store their files on Cloudfiles (We're kind of a file upload service in the most basic context). Switching in progress (to S3) and so far so good. I guess you get what you pay for, after all.
[+] groks|12 years ago|reply
You're having problems with uploads only? What about downloads?
[+] alecsmart1|12 years ago|reply
We were using Amazon CloudFront and bleeding money. Approximately spending about $50 every 2 days. I researched to find a cheaper alternative. And found cdn77.com. Very good and reliable. The costs came down to approx $100/mo. I am not sure about custom SSL pricing though.
[+] philfreo|12 years ago|reply
Bleeding money due to frequent cache invalidations, or traffic?
[+] dedalus|12 years ago|reply
Try www.instartlogic.com which is much better if you have dynamic content or image heavy apart from letting you not buy new servers thanks to the offload while providing great user experience
[+] cloudflare|12 years ago|reply
typerandom: if you have time I'd be interested in understanding the performance problems you reported on CF. My email is in my profile.
[+] typerandom|12 years ago|reply
Thanks. Would be happy really if we got it working with CloudFlare! /R
[+] randomhunt|12 years ago|reply
Please don't go to Cloudfront, it's truly horrendous (wearing my end user hat) to see so many sites have problems and timeouts because of them.

For example Ninite.com moved to them and now the site is virtually unusable because of it, over 90% of the time I get a "403: Forbidden" error when trying to do anything...like send them feedback on how the website doesn't work...

I even see more HN problems this week and I'm sure its because of them.

[+] asb|12 years ago|reply
Do you mean CloudFlare rather than CloudFront?
[+] captn3m0|12 years ago|reply
GitHub switched from Akamai to Fastly a little while back. I'd love to see a comparison of all these services.
[+] typerandom|12 years ago|reply
What is your experience? :) What CDN are you using for your startup today? /Robin
[+] Wouter33|12 years ago|reply
I have been using CDN77.com, you should check them out. Custom SSL is 699 dollars but the performance is really good.