eveningtree's comments

Also, keep in mind that the first month could be incredibly frustrating. (it was for me).

Once you understand the industry meta, until you learn the tools, it may turn into an exercise of frustration management.

But it is also a lot of fun.

All the very best.

Data engineering and systems design are incredibly close. After some time, they completely mesh together.

And from personal experience, the knowledge-volume to become practically potent is actually smaller from data engineering side (than from normal software development side).

I've found it easier to get data gigs and simultaneously strengthen the knowledge.

(I love databases, so that's also a factor)

The bonus $$$$ is also a nice side effect.

(It is pretty hands on.)

After you are done with the initial learning, find an academic machine learning discord or something similar. There will always be people there who will be very happy to find someone to clean their data. It's a great way of getting hands on with data engineering.

System design is best learnt through fires.

A good angle of attack is: pick a certification like AWS SAA or equivalent(AWS, azure, Google cloud. Doesn't really matter. Just pick a mid level certification). Then do the labs. They will quickly point out the holes in your knowledge/understanding. The free tier will take care of your needs, and cloud providers most of the time forgive the first surprise bill(it happens to everyone).

Soon data engineering and systems design merge anyways. This path is like a cheat-code for forcing convergence. Otherwise, the mind will quickly forget most theory.

---

Adding to this:

When you get hands on, you will find how the mind lies to you about how much you know. Mind maps are a nice way to reliably detect holes in understanding. Sit down with paper & pen once a week or so, and make a big map of everything you know.

Web development is hard. The individual parts are manageable, but bringing them together is a mess. We web developers are aware of this. It's like an open secret.

And it takes time getting comfortable with the mess, and letting the brain develop its own map of how things work. And you will do stupid mistakes along the way. Those are funny actually.

I'll give you my current perspective:

I've been at this for a decade, and I am definitely not incompetent. Last week I spent the night fixing a seemingly simple bug. It ate up my night and sanity. I went to bed angry. Woke up, are realized that I was looking at the right service, but editing the files in the wrong folder. Of course I didn't see the output. And I couldn't stop laughing.

This sort of silly mistakes happen almost once every month.

It can be frustrating, it will be. In webdev, things are in constant flux.

But try looking at it through a sense of humor, and you will see how funny it all is. The mentality will carry you through the decade.

We are just anxious & weak monkeys, with big brains, worrying about glowing glass powered by stone with lightning. It's surprising we even made it this far.

You and I deserve the leeway to make some mistakes for ourselves.

"why's (poignant) Guide to Ruby"

All right, this one is not purely technical. It's technical, but mixed with comics, art and a lot of personality.

It is an old classic in the community, and something that I aim up towards. Opened up my imagination to what a unique thing a technical book can become.

Here is how it feels: the brain experiences stuff "objectively".

For example: if something happens and you get angry, instead of fully being angry, you get to see that you are angry. And now you get the choice to be angry or not. Or sometimes you don't get a choice, but can clearly see it and realize that it's something you have to go through. There emerges a clear separation between your brain, your emotions and the core you.

It's not something you do consciously, it's something that your brain does by itself all the time. Like fish noticing water. Things don't really change, but you get a perception to notice and see things.

An analogy I like to use is: You are playing a 2d game. Your character exists in a 2d world. But your perspective is 3d and can clearly see what's happening.

--------

P.S.

Since you seem to be curious, keep in mind that meditation is serious stuff and needs to be given respect. Using meditation for just feeling good is like using a gun to dig a hole. Can you do it? yeah. But without gun discipline, you may end up shooting yourself in the foot.

Meditation is a tool. It exists safely only within a larger ecosystem of other things. Prescribing meditation openly without the ecosystem is dangerous.

Once mindfulness comes, the problems get more and more subtle.

By the very nature of solutions: a new company cannot beat google at general search. Google was and is search.

Like a tree that has grown up and shadows the entire land around it. There will be random small pockets of sunlight with smaller plants in it. But the big tree owns the area.

There can be a short term competition via extremely specialized search engines, but they will not rise to the same dominance. Google, in a way, is the entire idea of searching the web.

The next dominant generation has to be from a new paradigm, that makes web search obslete.

---

A relatable current example could be: youtube being dwarfed by tiktok. The nature of the new thing is such that it very naturally dethrones the old thing, without directly competing with the incumbent.

There will be specialized providers like vimeo, But at this point, youtube IS the idea of video on the internet.

---

Just note the context of this understanding: This understanding came from struggling to change the education system of my country. And then it seemed to apply everywhere I saw.

I'll round-off my reply with this quote:

“You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.”

– Buckminster Fuller

https://youtu.be/aVwxzDHniEw

Found this via a contest by Grant Sanderson, and it absolutely blew me away(and most people I forwarded it to)

It simply worked for my old folderwise-kept mp3s etc, and it also displays embedded lyrics in the mp3 files which was a killer feature.

It's not that I love this app dearly, it's just that it got the job done and didn't need replacement.