[swimwiththebeat]

How did you even get started with this? There are many systems I'd love to learn to implement from scratch like databases, network stacks, and caches, but the task seems so vast and daunting that it's hard to get started. Learning from existing codebases is also difficult b/c there's so much code to go over and understand. Can you elaborate on what your process was to build this without any help?

[vlovich123]

The strategy that works best for me is to just start at the simplest step & iterate from there. Do research online to find blog posts / articles describing how to solve specific problems. When doing reading, make sure to note terms of art that repeat so that you know what to look for.

So the first step I would do to start on a TCP/IP stack would be "how to open a raw socket" (granted you have to know the magic phrase here). There's also tons of "how to implement TCP stack" articles (same for ethernet etc) that probably are good starting points if I didn't know that magic phrase. The other thing is to find people to talk to who know more than you to answer those questions.

Once you have that, you can setup a real TCP socket on one end and a raw socket client on the other & then the same thing in reverse. Then look up the TCP/IP framing on Wikipedia (+ read up on the broad overview of how TCP works and the various parts). Once you have the framing implemented, try implementing the basic sequence to establish a connection. Then keep noting what features a full TCP stack has, which are required, & which are missing from my implementation (that's when I'd start reaching for the standards that everyone references).

Of course, if you want to do something novel beyond just "hey I have confidence that I can implement such a stack myself" (e.g implementing something with certain performance characteristics) that requires a deeper understanding of how things work and a good filter on possible ideas & which ones are going to likely work out the best (that is gained through expertise, creativity & intelligence)

[pipes]

If you made this into a book, I'd buy it! Especially if it was structured so that each chapter had a coding exercise that built on the previous one so that by the end I had a working stack.

My favourite programming book is structured like this "elements of computer systems" also referred to as "from nand to Tetris"

[cozis]

I think it boils down to the ability of being able to divide a big problem into smaller parts and understanding in what order you need to takle them. The more you try building big things the better you get at doing it.

After getting into university I decided to build an interpreter[0]. For someone who didn't even have the notion of a parser, it just felt like an unaproachable task. Even though, I sticked to it and the architecture became clear in time. That's another thing, even if you try to build something and miserably fail at it at each iteration, you still get better at it. The feeling of the task being too vast and daunting is just a feeling and knowing there's something on the other side makes it easier to power through it.

Hope to see your database on HN someday! :^)

[0] https://github.com/cozis/Noja

[technothrasher]

I wrote a little embedded TCP stack a while back as a learning project. I just read the RFCs and coded away. I'm sure it's the world's least efficient stack, but it wasn't too hard to get basic functionality.

[Lukeisun]

I am not very familiar with this kind of stuff either but I have read through "Beej's Guide to Network Programming" https://beej.us/guide/bgnet/html/split/. Which seems like enough to implement something like OP posted.

As for db's/other interesting things, I haven't read them myself but this site seems solid https://build-your-own.org/. If anyone has any real experience with this site, I would love to hear it!

[coddle-hark]

No OP but I think there’s three parts to really getting TCP/IP:

- Read the spec(s)

- Learn the OS APIs deeply

- Look at a bunch of real network data in e.g. Wireguard

[OJFord]

> Look at a bunch of real network data in e.g. Wireguard

I think you probably meant data in e.g. Wireshark?

But another good suggestion (or maybe what you meant) might be to look at 'real' (production) networking code in e.g. Wireguard, I imagine.