I have 10ms ping to news.ycombinator.com, and 100ms ping to www.amazon.com. Yet time to first byte is 20% faster to www.amazon.com. What actually happens is my PC connects to Cloudflare, witch in turn connects to HN. This is an unnecessary step, and is highly over-rated.
It's needed because CDNs are presently 'protection rackets' for the Internet.
Instead of having a mechanism by which a website under attack (or simply un-monitored heavy load) from a host or number of hosts can direct the ISPs of those hosts to not send it traffic the CDNs instead use the present lopsided nature of peering agreements to simply sink the hits.
In the above mentioned solution either the direct ISP would filter out the requests before they hit a higher tier ISP/backbone, or an ISP that does not provide said filtering would (possibly blacklisting the entire client ISP for sufficient bad behavior).
Picking amazon isn't really a fair comparison, It's Amazon, the people behind AWS, the same people who say that "every 100ms of latency costs them 1% of profit." Of course amazon.com and its servers are ultra-optimized to make everything as fast as possible.
A fair comparison would be hacker news with vs hacker news without cloudfare?
Wait, are you arguing that using a CDN is overrated? For everything, or just for HN?
A CDN is extremely valuable for a website that has cacheable content, since it allows a site to scale dynamically with increased load. This is the same reason people use AWS or any cloud provider; a website has to have the capacity to serve their highest peak traffic, but it is needlessly expensive to maintain that much infrastructure around the clock.
Helps a ton if you're terminating SSL at the edge, due to the number of roundtrips you need before the server can even start generating that first byte.
> This is an unnecessary step, and is highly over-rated
Given the volume of traffic HN sees, and given their desire to not be DDoS'ed, it seems to be a necessary step, that is not over-rated.
To be clear, you'd see similar results with any CDN sitting in front of HN... The benefits really start to show (from an end-user's perspective) when you're on poor connections or are far away from their (HN's, in this case) server/cluster. HN sees immediate benefits for the things above, plus it saves them a lot of bandwidth.
Yeah absolutely this. They spin everything they do as some kind of heroic "for the people!" decision even when it's just about cutting costs or not having to solve "hard" problems. One example are DNS "any" queries. Cloudflare just decided to toss standards out because they aren't up to conforming to them. As far as I'm concerned, this Cloudbleed thing is karma, and nobody should believe anything Cloudflare says about itself.
To be honest though, ANY is mostly used for reflection/amplification attacks or to scrape domain records. Sure there are legitimate uses for them but I can't think of many that need to happen over the internet vs. only allowing that from specific trusted parties. And yes I'm aware that it can be useful as a debugging tool. There are also ISP recursors that don't allow ANY queries through for similar reasons so relying on it will cause trouble for some and there are other broken implementations in the wild. Though ANY can be useful it shouldn't be assumed that it'll work.
What I'm thinking they could've done in the case of ANY is to respond with truncated and switch to TCP mode at which point it becomes harder to do the spoofing dance and (ab)use ANY as a DNS amplification attack to DDoS a target. Unfortunately that does put an extra cost on the DNS server which might also be undesirable. However that would've probably been a worthwhile tradeoff.
Lovely bit of debugging in this article, I really enjoyed it! Somehow a task that would be grueling to do myself is so much more enjoyable when read about.
We need to switch dump pipes to be dumb content-addressable p2p pipes with maidsafe, ipfs, dat, or anything like that, and most problems that CDNs are trying to solve would disapear.
[+] [-] z3t4|9 years ago|reply
[+] [-] mjevans|9 years ago|reply
Instead of having a mechanism by which a website under attack (or simply un-monitored heavy load) from a host or number of hosts can direct the ISPs of those hosts to not send it traffic the CDNs instead use the present lopsided nature of peering agreements to simply sink the hits.
In the above mentioned solution either the direct ISP would filter out the requests before they hit a higher tier ISP/backbone, or an ISP that does not provide said filtering would (possibly blacklisting the entire client ISP for sufficient bad behavior).
[+] [-] vmarsy|9 years ago|reply
A fair comparison would be hacker news with vs hacker news without cloudfare?
[+] [-] rdl|9 years ago|reply
If you tried the same thing with an image-heavy site, you would probably see the opposite result.
[+] [-] cortesoft|9 years ago|reply
A CDN is extremely valuable for a website that has cacheable content, since it allows a site to scale dynamically with increased load. This is the same reason people use AWS or any cloud provider; a website has to have the capacity to serve their highest peak traffic, but it is needlessly expensive to maintain that much infrastructure around the clock.
[+] [-] fidget|9 years ago|reply
[+] [-] Alupis|9 years ago|reply
Given the volume of traffic HN sees, and given their desire to not be DDoS'ed, it seems to be a necessary step, that is not over-rated.
To be clear, you'd see similar results with any CDN sitting in front of HN... The benefits really start to show (from an end-user's perspective) when you're on poor connections or are far away from their (HN's, in this case) server/cluster. HN sees immediate benefits for the things above, plus it saves them a lot of bandwidth.
[+] [-] fapjacks|9 years ago|reply
Yeah absolutely this. They spin everything they do as some kind of heroic "for the people!" decision even when it's just about cutting costs or not having to solve "hard" problems. One example are DNS "any" queries. Cloudflare just decided to toss standards out because they aren't up to conforming to them. As far as I'm concerned, this Cloudbleed thing is karma, and nobody should believe anything Cloudflare says about itself.
[+] [-] daenney|9 years ago|reply
What I'm thinking they could've done in the case of ANY is to respond with truncated and switch to TCP mode at which point it becomes harder to do the spoofing dance and (ab)use ANY as a DNS amplification attack to DDoS a target. Unfortunately that does put an extra cost on the DNS server which might also be undesirable. However that would've probably been a worthwhile tradeoff.
There's an IETF draft that attempts to provide some guidance in the area of the ANY query: https://tools.ietf.org/html/draft-ietf-dnsop-refuse-any-04
[+] [-] syncsynchalt|9 years ago|reply
[+] [-] dpc_pw|9 years ago|reply
[+] [-] kevinr|9 years ago|reply
[+] [-] lanius|9 years ago|reply
[+] [-] edoceo|9 years ago|reply
WebCore is doing what it's told, just just being told very stupid things (very, very quickly)