top | item 26056969

(no title)

stack_underflow | 5 years ago

I feel like I'm on one extreme of this spectrum which is: I 'need' to develop a deep understanding of systems from the bottom-up and only then can I feel comfortable building on top of it/abstracting it away/being productive with it.

When I started learning socket programming I didn't feel completely comfortable with using it until I had a complete understanding of the networking stack from the PHY upward. I even needed to write my own simple ARQ implementation to feel like I could fully appreciate and have a "feel" for what two machines are "actually doing" when they're sending packets back and forth.

I had the same obsessive desire to completely understand from the bottom-up how database engines were implemented because I just didn't feel comfortable with just the knowledge that "selects on tables with indexes on the field being queried are 'fast'". I just _had_ to know what was making them fast. Funny thing, when I found out that databases and database indexes were just disk-persisted/in-memory self balancing search trees or hash tables, I thought to myself, "that's it? why didn't anyone just tell me that instead of just repeating "queries are fast/optimized by the db"?"

I don't know if my brain just works differently because it seems most people I've worked with seem to get frustrated/annoyed with these details and not even have any desire to learn a system in-depth like this. They seem to only want to reason about a system at its highest level of abstraction, whereas to me, with the database example, once I learned that B+ trees were one of the primary data structures that backed database systems, I felt like the system as a whole just became that much more transparent and easy to reason about because now I could leverage all of my intuitions+past experiences relating to performance of operations on tree structures, having an intuition of perf implications of using linked data structures, reasoning about issues like paging chunks of data in and out of disk, etc.

discuss

order

Raidion|5 years ago

You just have an interest in different parts of the stack. Someone could write the same thing, but instead talk about why they needed to understand how the computer hardware was implemented to run the programs and even needed to build a small computer themselves to get a feeling for what the computer was "actually doing".

Alternately, at a systems design level, the constraint may not actually be the implementation of the components, but the organization and production of the system. This could be a simple CRUD app, where the business domain is more important, or the manufacturing of a new toy, where cost retooling and speed of production matters significantly more than trying to make the toy of a slightly higher quality.

There is a LOT of complexity out there in the world, and most people (on hacker news especially) gravitate towards parts of that complexity that interests them. A lot of other people gravitate towards different problems, which is a good thing, because it takes all sorts of specialties to get something nifty off the ground.

stack_underflow|5 years ago

That's a really good point. I've also done what you mentioned in your first paragraph but stopped digging deeper once I hit the "now I have to understand physics to actually understand why electricity behaves the way it does in order to flow through transistors/logic gates" barrier. My reasoning at the time was a combination of asking myself how much time I actually had to dive into that as well, plus how much of a potential return I thought I could get from having resolved those questions.