top | item 17529780

How to scale a distributed system [pdf]

214 points| HenryR | 7 years ago |cdn.oreillystatic.com | reply

10 comments

order
[+] matachuan|7 years ago|reply
Another quick hint: always make things work first then profile and optimize it. This happens when my colleagues were building a stream processing engine that scales up. At first, they start with all those fancy fine-grained locks but it turns out the final result just does not scale at all. They have to retreat to the version with some big locks and then figure out what was happening and how to solve it.
[+] hardwaresofton|7 years ago|reply
Am I the only one that found that there wasn't really much actual actionable information in here? There's basically no specific information about how they scaled along the axes they established were important
[+] HenryR|7 years ago|reply
That’s totally a fair point. I could have given a more concrete talk about how I’ve done these things in real systems (and the original version of this talk, presented internally to my company, had more real details and spoke very candidly about mistakes that I’d made along the way). Here I wanted to give more of a “here’s how you might want to structure your thinking, along with some basic design principles” kind of talk - I wish I’d had time to give more detail!
[+] scarface74|7 years ago|reply
I think of high level talks like these emphasizing timelessness. Technologies change over time and with in different contexts. But the theory remains as a way of thinking abstractly about a problem.

Besides, at the level of jobs I apply to now, interviewers care much more about my thought process than specific technologies.

[+] jhabdas|7 years ago|reply
I did a screen cast for these guys once. The majority of questions I got in reasons was where's the video.
[+] lalwanivikas|7 years ago|reply
anybody has a link to the video? google gives me only 3 min preview to the video :(
[+] agacera|7 years ago|reply
I think it is only available through Safari