That should probably read "doesn't scale without centralization".
The full answer to that question is way, way, way beyond the appropriate length for an HN comment.
DHTs need to accept requests from any random IP address, anywhere on the Internet. If you allocate state, you have to deal with resource exhaustion attacks. Dealing with them turns out to be way harder than anybody thinks when they first start down this path.
Cloudflare and Google can use stateful HTTP(S) because they are so massive. They can employ hordes of people to constantly tune DOS filters to block griefers before state allocation occurs, and lawyers to deal with any griefers who are as large as they are. If you're happy with that kind of centralization you shouldn't be wasting your time with DHTs; you don't need them.
Aside from resource exhaustion attacks, there are some pretty deep lessons connected to systems and architecture. The bittorrent bootstrap nodes serve some really insane number of requests/second on very modest hardware (I'm still trying to find the link that detailed this...). The Internet root zone DNS servers were, for a very long time (and perhaps still today in one or two cases) just a few good workstations at universities.
Sessionless protocols scale like crazy, on cheap hardware. But they are very hard to design correctly. If you have VC bucks, spend them on hardware and use sessionful protocols.
You can restrict DHTs to accept well-formed requests and further restrict them using a web of trust. With a large enough network and an on onboarding path using nodes that have higher bandwidth, it can work to mitigate that vector I’d think.
It would be really hard to launch that attack with a DHT that implements appropriate pruning, drop, and block rules for bad behavior. All that can feed into a personal (& autonomous) web of trust. Any references for me to read more about this?
KirillPanov|3 years ago
The full answer to that question is way, way, way beyond the appropriate length for an HN comment.
DHTs need to accept requests from any random IP address, anywhere on the Internet. If you allocate state, you have to deal with resource exhaustion attacks. Dealing with them turns out to be way harder than anybody thinks when they first start down this path.
Cloudflare and Google can use stateful HTTP(S) because they are so massive. They can employ hordes of people to constantly tune DOS filters to block griefers before state allocation occurs, and lawyers to deal with any griefers who are as large as they are. If you're happy with that kind of centralization you shouldn't be wasting your time with DHTs; you don't need them.
Aside from resource exhaustion attacks, there are some pretty deep lessons connected to systems and architecture. The bittorrent bootstrap nodes serve some really insane number of requests/second on very modest hardware (I'm still trying to find the link that detailed this...). The Internet root zone DNS servers were, for a very long time (and perhaps still today in one or two cases) just a few good workstations at universities.
Sessionless protocols scale like crazy, on cheap hardware. But they are very hard to design correctly. If you have VC bucks, spend them on hardware and use sessionful protocols.
survirtual|3 years ago
It would be really hard to launch that attack with a DHT that implements appropriate pruning, drop, and block rules for bad behavior. All that can feed into a personal (& autonomous) web of trust. Any references for me to read more about this?