top | item 20794861

Things I Learnt from a Senior Software Engineer

849 points| neilkakkar | 6 years ago |neilkakkar.com

294 comments

order
[+] OJFord|6 years ago|reply
> in my team culture it’s not frowned upon to “snoop behind” people writing code. Whenever I sensed something interesting going on, I’d roll around and watch what was happening.

Agh, I'd hate that. In fact if anything interesting had been going on on my screen it would immediately stop, no way I could work with someone watching.

I struggle enough at desks with my back to a door or where people walk by, just can't stop myself from being distracted and looking.

(It's not that I'm not working, or otherwise doing anything I shouldn't be, it's just.. distracting is the best word I have. It's not where I'd sit in a restaurant, and I've despised not having a choice but to sit at such a desk at work.)

[+] teddyh|6 years ago|reply
> Naming your clusters? Naming them after the service that runs on them is great, till the point you start running something else on them too. We ended up naming them with our team name.

This is covered by RFC 1178¹, Choosing a Name for Your Computer (from 1990):

Don't choose a name after a project unique to that machine.

A manufacturing project had named a machine "shop" since it was going to be used to control a number of machines on a shop floor. A while later, a new machine was acquired to help with some of the processing. Needless to say, it couldn't be called "shop" as well. Indeed, both machines ended up performing more specific tasks, allowing more precision in naming. A year later, five new machines were installed and the original one was moved to an unrelated project. It is simply impossible to choose generic names that remain appropriate for very long.

Of course, they could have called the second one "shop2" and so on. But then one is really only distinguishing machines by their number. You might as well just call them "1", "2", and "3". The only time this kind of naming scheme is appropriate is when you have a lot of machines and there are no reasons for any human to distinguish between them. For example, a master computer might be controlling an array of one hundred computers. In this case, it makes sense to refer to them with the array indices.

While computers aren't quite analogous to people, their names are. Nobody expects to learn much about a person by their name. Just because a person is named "Don" doesn't mean he is the ruler of the world (despite what the "Choosing a Name for your Baby" books say). In reality, names are just arbitrary tags. You cannot tell what a person does for a living, what their hobbies are, and so on.

1. https://tools.ietf.org/html/rfc1178#page-2

[+] Spooky23|6 years ago|reply
Naming standards are the ultimate bikeshedding event. Everyone has an opinion and camps develop for various schemes.

The last time this happened, I happened to be in a position of influence for the final decision for naming standards. We took an approach designed to piss off everyone... license plates. We used sequential numbers prepended by a pronounceable string, and random words, selected by a system, for internal services.

Overall, everyone is pretty happy with it. YMMV.

[+] one-punch|6 years ago|reply
There is a cute name for, er, the two ways of naming systems: pets naming vs cattle naming [1]. I think naming after services is like pet naming, while naming them by numbers is like cattle naming.

Pets Service Model

In the pets service model, each pet server is given a loving names like zeus, ares, hades, poseidon, and athena. They are “unique, lovingly hand-raised, and cared for, and when they get sick, you nurse them back to health”. You scale these up by making them bigger, and when they are unavailable, everyone notices.

Examples of pet servers include mainframes, solitary servers, load balancers and firewalls, database systems, and so on. Cattle Service Model

In the cattle service model, the servers are given identification numbers like web-01, web-02, web-03, web-04, and web-05, much the same way cattle are given numbers tagged to their ear. Each server is “almost identical to each other” and “when one gets sick, you replace it with another one”. You scale these by creating more of them, and when one is unavailable, no one notices.

Examples of cattle servers include web server arrays, no-sql clusters, queuing cluster, search cluster, caching reverse proxy cluster, multi-master datastores like Cassandra, big-data cluster solutions, and so on.

[1] https://medium.com/@Joachim8675309/devops-concepts-pets-vs-c...

[+] shagie|6 years ago|reply
The thing I found when naming things was to have as much information packed in the first syllable in a way that was reinforced by all the remaining syllables (that becomes more important when there are a variety of accents in the workplace).

One set of systems I worked on, a team had names based on islands. Two of the test machines were Trinidad and Tobago. Upon hearing the first syllable, it was clear that these were the test machines. Every additional bit beyond tri or tƏ was distinct and reinforcing that the first bit was heard correctly.

They also had boring names like dc2f1a3r1s4 and foo-test1 (depending on who was interested in the machine).

The problem that I've found with names like asfcap1234 is that one has to listen to every piece of information and remember it to properly identify the machine. With every character being important and distinct, there isn't any reinforcement or mnemonic checksum to make sure you've got it right.

I will note that this is from the era of servers (and even before VMs started being the thing).

It is very true that once humans no longer reference the machines by name, the ability to name them becomes less important.

As a counterpoint to RFC 1178, consider RFC 2100 - The Naming of Hosts https://tools.ietf.org/html/rfc2100

[+] philwelch|6 years ago|reply
If you designate your servers with numbers or meaningless identifiers instead of names, you also won't ever feel guilty about killing and replacing them. I believe this is (somewhat gruesomely) called the "pets vs. livestock" approach.
[+] stefs|6 years ago|reply
in my home network i name my computers after animals, whereby the power/size of the computer roughly resembles the size of the animal.

my beefy desktop may be the whale, the nas is the rhino, the notebooks are roughly dogs, the raspberry pis are small animals like mice, the chromecast is e coli.

plus good: i'm never going to run out of animal names.

[+] jaggederest|6 years ago|reply
For some reason nobody liked my norse pantheon names.

Teaching people to spell yggdrasil correctly in order to get a database connection is fun. For some values of fun.

[+] rileymat2|6 years ago|reply
I am not understanding why the machine name could not be changed when it's purpose changed? Is an immutable name unique to that environment?
[+] kristopolous|6 years ago|reply
Names are an opportunity for fun. Call that old server PersistentSnail, or some old fashioned name like Myrtle. Make naming a reason to smile.
[+] grogenaut|6 years ago|reply
I find that team names change more frequently than responsibility in a large org. Also using a team name makes people assume ownership of decisions with that team. That's why I prefer service name. However I haven't thought much about shared use clusters.
[+] bluedino|6 years ago|reply
The only machine we have that's not named poorly, is named DP2950. It was originally a Dell PowerEdge 2950, and we probably had more than one back then, so maybe it wasn't a great name.

Everything else is named based on it's purpose.

www, webapps, data, acccounting, mail, things like that.

But, mail is actually named mailV3 (it's the third hardware the mail server has been on). Why it wasn't just named mail, and if the old one was kept around, it could be mail-old, I don't know.

Then we have our development servers. devel, develv2, www-devel, that kind of stuff. I'd personally prefer devel-whatever the name of the production version is called. Instead we have to thing 'develv2 is actually the development version of dp2950....'

[+] dotancohen|6 years ago|reply

  > Nobody expects to learn much about a person by their name.
This is wrong. If I tell you that I just hired a mathematician, a farmhand, and a nurse, you could probably guess which person was which when I introduce you to Vladimir, Jebbediah, and Sofia.

Names are a reflection of culture, and certain cultures tend towards certain professions. No politically-correct dreams will change that unless we instill a universal monoculture throughout the world. I'm sure that Disney and Coca Cola would love that, but most people would probably prefer to protect their heritage and culture.

[+] pure-awesome|6 years ago|reply
> It is especially tempting to name your first computer after yourself, but think about it. Do you name any of your other possessions after yourself? No. Your dog has its own name, as do your children. If you are one of those who feel so inclined to name your car and other objects, you certainly don't reuse your own name. Otherwise you would have a great deal of trouble distinguishing between them in speech.

a) I am, in fact, named after my father.

b) This does, indeed, cause trouble for other people trying to differentiate us.

[+] paulsutter|6 years ago|reply
Like the inevitable branch names. Main, Borg, Borgmain, etc... Or the purposes of the standard unix directories (lib, usr, opt, etc, ...)
[+] galaxyLogic|6 years ago|reply
Isn't Cloud-Computing something that should make it unnecessary to name individual computers?
[+] amzans|6 years ago|reply
> Good engineers themselves design systems that are more robust and easier to understand by others. This has a multiplier effect, letting their colleagues build upon their work much more quickly and reliably - How to Build Good Software

The advice above is probably one of the things which would have the most impact across all your activities as a developer. When people talk about simplicity in software, they don’t necessarily refer to ease of use or number of lines of code, but instead it’s about how understandable a solution is given their shared knowledge.

[+] Everlag|6 years ago|reply
The referenced idea of a 'human log' is great[0]. I started doing something similar 4 years ago and it eventually evolved from per-project notes into a full diary. Being able to search for 'August 24 2016' and know exactly what I did that day is quite powerful.

I encourage anyone to take 10 minutes(or 30...) at the end of the day to write up what they've done. Just a text file with minimal formatting has scaled to 2.6MB of hand-typed text. Though, after a bit, I've tended to shard out specific long-running topics into their own files.

[0] https://neilkakkar.com/the-human-log.html

[+] noonespecial|6 years ago|reply
I was already forgetting things I learnt. They either became so internalized that my mind tricked me into believing I always knew them, or they slipped my mind.

The best time to write a tutorial is as you learn a thing yourself. You learn it better for doing it and others benefit from it more because it automatically comes from a beginners perspective.

[+] caymanjim|6 years ago|reply
> I like a bit of humour in my code, and I wanted to name it GodComponent.

No. Just no. Do not ever be funny in code. No one else likes your humor, and it's distracting. (Ignoring the other reasons "GodComponent" is a bad name.)

[+] cwbrandsma|6 years ago|reply
I needed to wrap the code that handled thread management that I use in Swift. I called the component Threader, and in the comments I mention I named it after the TMNT character Shredder.

Two things came from that: I have gotten a chuckle from all the developers I introduced to that code, and no one has forgotten about it.

Some humor is useful.

[+] askafriend|6 years ago|reply
Meh, I don't value that opinion at all. Especially a hard-line stance of "DO NOT EVER".

I'm going to continue injecting some light humor into my code and it'll be fine (especially unit test code).

[+] philwelch|6 years ago|reply
I think it's okay to be funny in test cases. For instance, my test data has included the user-agent string, "The Thrilla In Mozilla".
[+] newshorts|6 years ago|reply
I agree on the aspect of professionalism. I wouldn’t go so far as to say never. Maybe there is some situation out there where it’s appropriate and understandable. I’ve just never seen it ;)
[+] heinrichhartman|6 years ago|reply
> When refactoring and preventing huge-ass PRs: “If I’d have changed all the tests first then I would have seen I had 52 files to change and that was obviously gonna be too big but I was messing with the code first and not the tests.” Is breaking it up worth it?

My 2 cents: There are two things to consider:

1. reviewability

2. deployment risk

If it takes a colleague 3 days to review your code, your PR is too big. If you panic, on the thought of deploying this ever, your PR is too big.

On the other side:

- Huge PRs that only change formatting are fine. Easy review. Low risk if properly automated/tested.

- Largs PRs that are feature neutral are acceptable as long as they are reviewable.

- PRs that refactor 2000LOC, fix 2 bugs and add 3 features are not a good idea.

[+] foobarian|6 years ago|reply
There are certain ways to organize large PRs to make them easier to review and deploy. Some examples:

- if a large refactor moves code around in addition to changing logic, omit the pure moves from the diff and provide a list of files moved in a description

- if a large refactor involves a lot of renaming, or other mechanical manipulation, in addition to a little bit of logic changes, split those apart. That way there will be one large mechanical diff, where you can state in the description, "Yes I changed 500 files and there are 20 pages of diffs but the nature of changes is the same as in the first file." This is vastly easier to review for your peers.

- A useful safe deployment trick, when feasible: guard new functionality with a kill switch, as well as some kind of sampling gate (e.g. use new logic only where customer id % 31 == 0). Then when proved remove old code in a followup.

[+] einpoklum|6 years ago|reply
The thing is:

1. Some people balk at you making lots of small PRs, because it feels like noise to them. 2. Some people/organizations make you do a lot of work on a separate branch and then want you to PR just once for the final feature you're implementing.

[+] honkycat|6 years ago|reply
Just to toss in my two cents about testing:

Unit tests are for refactoring and declaring behaviour. Additionally, a lot of people have made the same observation: Code that is easy to unit test tends to be more modular and have better architecture. If you are struggling to test a function, think about how you can change your design to make testing easier. It will probably improve your code quality.

Integration tests are for finding bugs and exceptions and should be a part of CI/CD.

You want both. The more testing the better.

Comments: I find comments, outside of docstrings, a smell. Even then: Docstrings often become a substitution for a decent type system, so maybe think about what is happing there as well.

Documentation should be generated from your codebase. Otherwise, it will inevitably become out of date as people forget to update it. If you need to comment on what something is doing, usually you can move that behaviour into a function and test that function.

[+] Ididntdothis|6 years ago|reply
I think this is pretty good.

As a pretty senior dev myself I always tell people to have an actual design in mind. Know what you are trying to achieve and when something goes wrong you should be able to explain what you expected the system to do on that situation . A lot of people don’t seem to have a clear mental image of the end goal but are overwhelmed .

Of course you should be flexible but you need to know where you want to be in order to make good decisions.

[+] ulkesh|6 years ago|reply
I like the article. I just wish the senior software engineer also included teaching humility. The article isn't written with any ego, so I commend the author, but it would be ideal for any mentor to also teach what to do when being wrong, making a mistake, costing the company money, etc.
[+] contingencies|6 years ago|reply
The main value in software is not the code produced, but the knowledge accumulated by the people who produced it

Don't agree with this at all. If you're relying on people to maintain knowledge then you're doing it wrong and setting yourself up for failure. Document the why.

[+] tehlike|6 years ago|reply
You are thinking just the running system. The decisions behind why, the quirks of the software, how that all fits together, and the knowledge of why you should do certain things (like monitoring, experiments and whatnot) is important.

Good luck documenting all that. Good luck making that document discoverable and readable.

[+] atoav|6 years ago|reply
accumulated doesn’t allow any qualitative judgement about the way they treat that knowledge. So we shouldn’t assume the worst.

I suggest the opposite: if they take their own words seriously and this knowledge is the main value than I’d assume they grow in great lenghts documenting that main value — because it would be (as you rightly noticed) lost.

[+] BurningFrog|6 years ago|reply
The way I think of it, the knowledge should be in the code. Including the tests.

To me that is one of the major goals of any code writing. Some new hire I've never met should have a fair chance of working with this in 6 months.

[+] X-Cubed|6 years ago|reply
My take on this point (and something I agree with) is that you shouldn't get too attached to a particular implementation. It can be useful to build a prototype and throw it away, then use that knowledge to build a better version, than to try to keep building on top of the prototype.

There's nothing in their point that refers to documentation.

[+] lonelappde|6 years ago|reply
Parent didn't say what you claim to disagree with. The documentation is how the knowledge is accumulated. People maintain documentation.
[+] ztjio|6 years ago|reply
Makes me a little ill that the author thinks Jeff Atwood coined that old-ass joke.
[+] neilkakkar|6 years ago|reply
.. Nopes, but it was the quickest reference I could find for the joke. I don't like to put quotes in without the source.
[+] chaboud|6 years ago|reply
“De-risking is the art of reducing risk with the code that you deploy.

What all steps can you take to reduce the risk of disaster?”

Honestly, I talk about derisking most in two contexts in our projects: schedule, invention.

Normally this means that derisking is about frontloading unknowns and finding pivot decisions that reduce the likelihood of getting too far in the wrong direction or accumulating too much uncertainty.

I don’t use “risk” too often when talking about fault tolerant code or system design. If you’re taking on “risk” in code, like the chance of a lost message, a race condition, or corrupted data, I’m going to take away the keys.

Note: I encounter shoddy “we can go hours without a crash, so it’s good” work more than I’d like to admit. It is possible to know why code and systems fail (and account for it) much more than has become industry norm these days.

[+] 7532yahoogmail|6 years ago|reply
This article is fluff. Presented with a gosh-this-is-cool-isnt-it narrative, nothing here suggests senior engineer. Senior engineers must have a much more nuanced assessment about the interaction between organizational norms, teamwork, and software. There's no penetrating insights here. Software is semi-formal at best and more a hidden Markov chain at best. Indeed the central questions to software engineering are deferred as questions.
[+] xuesj|6 years ago|reply
Good programmer should be good at writing essay.
[+] greyskull|6 years ago|reply
> In the end, we went with a database with role access control (only our machines and us can talk to the database). Our code gets the secrets from this database on startup.

How does the code get access to the database?

I've lived in AWS-land for so long, I don't know how the non-cloud world manages secrets.

[+] greyhair|6 years ago|reply
Requirements matter. Design matters. Data matters. Documentation matters. Source control matters. Defect tracking matters.

All source code eventually "wears out" and has to be rewritten.

[+] mleonhard|6 years ago|reply
> I’ve come to love testing so much that I feel uncomfortable writing code in a codebase without tests.

This person became a good engineer in only one year! :)