Rolling your own CDN for $25 in 1 hour

[natch]

Nice. For tutorials like this it would be nice to see more details on the "how" of each step. For example:

"We setup DNS info to point here (both NS1 and NS2..."

That's great, but it would be nice to see:

"Using the geodns web-based configuration tool, we setup DNS info to point here (both NS1 and NS2..."

or

"Running <insert full command line here including tool and all options used here> on <insert environment here>, we setup DNS info to point here (both NS1 and NS2..."

or

"By editing /etc/<file> on the <some environment>, we setup DNS info to point here (both NS1 and NS2..."

In other words specifics are always really great to include.

But still this was a really great writeup, not just for the overview it gives of one lightweight approach but for the comments it has kicked off.

[acoyfellow]

I think Genius and DigitalOcean = a match made in heaven. The quality would go from "the best" to "untouchable", if the comments were value adding. Imagine being able to dive deeper into any piece of the tutorial you wanted.

[joesavage]

I was planning to publish a post with literally the same title as this one later this week (annoying coincidence as I guess mine will now be largely ignored - maybe I should wait a few weeks to let this space calm down, I'm not sure) with some in-depth step-by-step explanation - so watch out for that :)

[fred_durst]

This comment is a model of quality feedback. It shows appreciation, not just shallow bookending, and gives detailed improvement examples. I know this is off topic a bit, sorry.

[davidcelis]

> Step 1: Order a DNS instance from Digital Ocean.

This also feels lacking to me. How does one order a DNS instance from DO? I can tell from the icon that it has something to do with CentOS but I don't see any sort of DNS server under their available applications or images for a one-click install.

[mxpxrocks10]

great points. happy to take this approach on future posts and would love to collaborate in the future.

[xxdesmus]

I'd recommend CloudFlare if you want AnyCast geo-DNS, and a free CDN. We don't charge for bandwidth or our DNS service.

Disclosure: yes, I work at CloudFlare but the previous details are simply facts.

[kawsper]

Do you have a link to CloudFlare geo-DNS? I have multiple servers in multiple parts of the world, and didn't know that Cloudflare supported that.

[tszming]

Hi, I heard that CloudFlare is now blocked in China, do you have more information?

[kbar13]

rolling your CDN on VPS instances isn't exactly cost-effective, and is probably more of an exercise for fun than anything else. Unless you need features out of your CDN that you can't get from existing CDN providers, I don't really see the point.

also kind of unusual that this blog post is by the president of MaxCDN.

[Oculus]

kind of unusual that this blog post is by the president of MaxCDN

It makes a lot of sense for the president of MaxCDN to make a blog post like this. Bob the developer decides to learn about how CDN's work and happens upon this blog post. Cool, Bob just built his own mini CDN network for fun. Now Bob understands that his CDN, though a fun weekend project, is definitely not production ready. What does Bob do when he needs a CDN he can put into production? He goes to the guys that established themselves as authorities on the topic: MaxCDN.

Content Marketing 101: http://www.kalzumeus.com/2012/02/09/why-i-dont-host-my-own-b...

[pushrax]

Actually, with DigitalOcean, you get quite a lot of bandwidth for the price. Additionally, their transfer limits are high (1TB for the $5 instances), and overage is only $0.02/GB. If you combine this with something like Route 53's anycast latency-routed DNS, it can work very well.

[mxpxrocks10]

hey. thanks for the feedback. Yeah, it's more of an exercise for fun but it can also be done w/ high bandwidth dedicated servers as well. It's good as well if it's heavily dynamic stuff that needs really low latency and you need to execute something custom on the edge. Otherwise MaxCDN.com or a similar service scales better. I'm looking forward to building more custom stuff into our edges at MaxCDN.com so feel free to throw out any ideas. This was used to create a presentation for a docker meetup.

[akerl_]

I'm more than a little bit concerned that the article doesn't point out the serious risk of running a single DNS server for the records.

I get that they're trying to keep things simple for their demo, but unsuspecting folks following this guide are in for quite a shock when their GeoDNS is taken out by a single physical server fault.

[mxpxrocks10]

Good point - it seemed obvious. We'll cover different options for that in a future blog post.

[mxpxrocks10]

added this to the article in the "Todo" section. thanks for the feedback.

[edoceo]

This is similar to the process I used to build http://gcdn.org/ which I and a few clients + friends use on multiple projects. We used to use RR but GeoDNS has proven to be awesome and reliable. Full NS control makes it easier to do rolling upgrades or maintenance too. Excellent write up.

More notes on GeoDNS http://edoceo.com/howto/geodns

[umsm]

This returns a 404-not found for me:

https://gcdn.org/jquery/1.8.0/jquery.js

[danielrhodes]

I would highly recommend Varnish using S3 as a backend origin. Add to that a few 100TB.com dedicated server and you have an extremely cheap CDN with pretty decent bandwidth and minimal maintenance.

CDNs have the advantage that they control more of the stack so can do more precise routing and have more edge nodes in more places. However, under the right circumstances, you can take the above quite far before a real CDN becomes necessary.

[RKearney]

100TB.com specifically forbids using their service to create a CDN in their ToS[0].

    9. Acceptable Use/Illegal Activity

    d. We strive to maintain a high level of service, and a lot of
    customers depend on our high standards of quality. As such, we
    will not provide Services to those that are using our Services
    for:
    
    vii Using the Services for a content delivery network or content
    distribution network (CDN). An authorized CDN network offered
    through 100TB is accepted. Special requests to use the Services
    to run an unauthorized CDN network may be approved on a case-
    by-case basis. Failure to comply with this policy will result
    in termination of this TOS, and you will not receive a refund
    of the Fees.

[0]: http://www.100tb.com/tos.php

[mxpxrocks10]

hey Daniel- thanks for the comment. Do you run any special varnish config settings w/ s3?

[AdamN]

Now with libcloud (http://libcloud.readthedocs.org/en/latest/compute/drivers/) you can use multiple platforms in order to get all continents represented.

Including Africa (http://kili.io) where I'm at.

[mxpxrocks10]

has anyone ever heard about these guys in South Africa: http://www.teraco.co.za/? Any other recommendations for Africa and South Africa?

[mxpxrocks10]

awesome. Thanks for posting. I had that as one of my todo's in the blog post. Feel free to reach out to me if you want to help with a follow up post!

[muriithi]

Why Africa and not Kenya?

[ColinCera]

I'm wondering if it would be useful to plug groupcache into this. It's what Google uses for serving downloads (e.g., Chrome downloads). Inherently distributed and self-balancing. I might give that a try.

[xin3td]

Looks promising, but might need to add Doozer to the stack as groupcache has an issue with maintaining a list of peers where the cached data is distributed. Definitely worth looking into though.

https://github.com/ha/doozerd

[mxpxrocks10]

great idea.

[fakeyfake]

Reminds me of this article about hosting a CDN on digitalocean for the dash documentation software.

http://blog.kapeli.com/a-poor-mans-cdn

[mxpxrocks10]

cool - thanks

[jpmonette]

Definitely nice to see a real-world example of how to use Docker.

[mxpxrocks10]

woot woot.

[Oculus]

Cool article for understanding the basics of a CDN. What sort of page load improvement can one get by adding geographically distributed servers?

[mxpxrocks10]

it can be quite a lot. you can test using a CDN or a setup like this and the tools over at www.webpagetest.org (from Patrick Meehan - who is awesome) from different locations.

[ksec]

Wouldn't it be better on Linode? Since they have wider selection of PoP. But i wonder if the port speed would be a problem.

[fmela]

Thanks for sharing. Note that there's a typo in the link to Ewan's github profile (should start with https, not ttps).

[mxpxrocks10]

thanks. will get it fixed shortly. this wasn't really scheduled for prime time but started taking off. appreciate the feedback!

[mxpxrocks10]

got this fixed. thanks for the heads up.

[adarshr]

Why is this page blocked by my UK work network?

[Kudos]

The people doing the blocking can probably better answer that.

[lingben]

or you know, if your concern is cost, go with cdn.net and pay as you go for a kickass global cdn network

full disclosure, I have no business or personal connection to them

[Kudos]

There's 6TB of bandwidth built into this home rolled solution. That's going to cost at least $350 at cdn.net.

[leapius]

Excellent tut!

[hiddenpearls]

that's awesome +Chris