Hi, what concepts do you think a backend engineer should know which are language agnostic/independent? An example could be knowing the different ways of how to connect to the database from the backend such as ORM or basic SQL and their pros and cons?
I think it's important to know about different data storage options and their trade-offs. Managing state is one of the hardest parts of backend development, particularly at scale, so an understanding of the trade-offs in databases/caches/blob-storage/queues, and when each is useful is important.
I'd pay close attention to speed and "correctness". What's the consistency model of a system? Can we lose data and if so how? What's the throughput? Latency?
These help choose good technology for backend systems, and helps answer questions like:
- Can we do this in-band while serving a user request?
- Can we do it 100 times to serve a request?
- If it completes successfully can we trust it or do we still need to handle failure?
- Can we trust it immediately or eventually?
There are lots of technologies and terms for all of this but I've specifically avoided them because the important bit is the mental model of how these things fit together and the things they allow/prevent.
Also, I’ve had to explain this to so many other engineers, both junior and senior to me: most data is inherently relational. This next statement is a bit opinionated: 9 times out of 10 you probably want an RDBMS. I’ve seen so many attempts to shoehorn some ElasticSearch/Mongo/Neo4j/whatever database into a design because the developer wanted to work on CoolDatabaseTech. Then you’re stuck dealing with joins in CoolDatabase that it wasn’t really designed to do and frustrated at CoolDatabase’s lack of drivers in X language. Later on you’re dealing with stability and scalability issues you would never see with BattleTestedRDBMS.
The amount of capability a well designed Postgres instance can output is insane. I’ve seen a single vertically scaled Postgres instance compete with 100+ node Spark clusters on computations.
I'm currently learning back-end coming from a front-end career, and I started reading "Designing Data-Intensive Applications" by Martin Kleppmann, seems to be a must-have for anyone who wants to get serious in this field.
+1 For this recommendation, an exceptional book. In particular chapters 2-3 demystify how databases work, and provides an intuition on when to use different kinds of databases and query languages without the typical hype (spoiler alert - use an RDBMS!)
Security mindset: read the book Security Engineering (it is online), less for specific technical info than for the many war stories etc. which will help you guard against vulnerabilities and unforeseen consequences.
Basics of cryptography: there are many dumb errors to avoid.
Thorough (not just basic) knowledge of SQL, if you don't count that as a language. The sqlite.org "technical and design documents" about sqlite's virtual machine and its query planner are well worth reading, and apply to other databases as well. ORM's are less important than SQL, and are usually language specific as someone mentioned.
Reasonable clue about socket programming, even if you're doing everything with libraries that wrap the details.
IAM, who and what is authenticating, how and what permissions will it have.
What data is coming into your system and it’s variety, velocity and volume.
Do you really need NoSQL, probably not.
Do you really need that ORM and all the schema, migrations and ops to go with it, known the pros and cons.
Are your boundaries defined well? Networking, firewalls etc? Are or do they need to be identity aware?
Are you logging what you need to log, where you need to log it and do the right people have access to it? Maybe metrics are really what you need.
What’s the dev story like? Can I run things locally or easily without spending days recreating an environment? IAC is one thing, but debugging some Python locally vs deploying and print statements sucks. Have a good readme and leave the repo better than you found it.
Tackle the hard problems first, then create reproducible developer story, then hand it off to someone Jr. so they can do the grunt work and you can help them out in a jiff.
CI/CD, incrementally improve it over time and don’t spend time boiling the ocean here. A simple bash script to deploy may suffice for an SRE to take it to the next step as IAC or to drop it into some CI tool.
Apply the practice of least privilege from the very start.
KISS, if what you are building is too confusing, it’s because you need to spend more time writing about it vs actually writing it.
If you understand the system a layer of abstraction or two below the layer you work in, you will be able to debug deeper. Learn system calls, Various ways how to examine processes.
I learnt a lot of this back in the day by completing war games on a site called digital evolution (dievo). Those are antiquated now but still a really fun way to learn it.
Also, learn Javascript and browser tech in general. One of the keys to high-performance systems is figuring out how to make the layer above and the layer below talk to each other while removing yourself from the process as much as possible.
IME, it's the following considerations that make back-end development hard:
- Fault tolerance.
- Backwards (or forwards) compatibility.
- Scalability.
- Testability.
- Everything around state (backup/restore, migration strategies, data integrity, etc.)
Most other things are a one-time cost. These things are an ongoing burden to consider, but if you forget to consider them it can be devastating.
Also remember: any time you give a (internal or external) customer programmatic access to something, that is an API, and APIs have huge costs to maintain. That includes when you dump your database into "data lake" for internal reporting...
- Authentication: OAuth2 is probably the most widely used
- Authorization: RBAC
- Some rudimentary statistics: know how to read metrics, write metrics, etc
- Learn one RDBMS inside and out. Other database systems have their place but you’ll almost always encounter a Postgres, MySQL, MSSQL. Learn how to read EXPLAIN output, cursor based pagination, and indices.
Understand the importance of having good visibility of your system. Implement good logging and collect metrics, for example the four golden signals of throughput, latency, saturation and error rate. The Google SRE book gives a good introduction to some of these concepts. See for example https://sre.google/sre-book/monitoring-distributed-systems/.
Understand how to load test your system and to reason about its behaviour under load and its failure modes when you push it too hard. It's one thing to be able to build a system and functionally test it such that you're confident that it behaves correctly when you send one request at a time. It's another thing to let thousands or millions of real users hit it for real in production and to have confidence that you are giving them all a good experience.
I would add soft skills - how to sell your ideas; how to ask for help, how to offer help with offering offense; how to write docs, emails, or proposals; how to avoid taking offense when it feels like you're being ignored or slighted (usually not the case); how to keep stakeholders updated; creating realistic timelines. I'm sure I'm missing some. You can accomplish more than your technically astute colleagues and work on more interesting projects if people trust you, like you, and feel inspired and happy to work with you.
Privacy management.. Privacy laws are severely restricting who may see people’s PII
Are you using the PII data for purposes other than it was originally collected?
Can you synthesise a good enough set of test data so you don’t have to anonymise production data? Hint: you can’t sufficiently anonymise production data and still have it be useful
I always thought the roadmap.sh Backend Developer Roadmap gives a nice overview of technologies and concepts. Even though it misses some things imo (identity management comes to mind)
I have found that the most important bit as a backend engineer is modelling the application domain precisely.
A taste for making good APIs helps and knowing how to measure performance.
These days most backend engineers also tend to manage data sources so understanding them is also a plus
Don’t hide the async nature of external requests. It seems like a magically good idea until it all falls apart. Timeouts and error propagation needs to flow from the request ingress all the way to the backend services.
danpalmer|4 years ago
I'd pay close attention to speed and "correctness". What's the consistency model of a system? Can we lose data and if so how? What's the throughput? Latency?
These help choose good technology for backend systems, and helps answer questions like:
- Can we do this in-band while serving a user request?
- Can we do it 100 times to serve a request?
- If it completes successfully can we trust it or do we still need to handle failure?
- Can we trust it immediately or eventually?
There are lots of technologies and terms for all of this but I've specifically avoided them because the important bit is the mental model of how these things fit together and the things they allow/prevent.
SOLAR_FIELDS|4 years ago
Also, I’ve had to explain this to so many other engineers, both junior and senior to me: most data is inherently relational. This next statement is a bit opinionated: 9 times out of 10 you probably want an RDBMS. I’ve seen so many attempts to shoehorn some ElasticSearch/Mongo/Neo4j/whatever database into a design because the developer wanted to work on CoolDatabaseTech. Then you’re stuck dealing with joins in CoolDatabase that it wasn’t really designed to do and frustrated at CoolDatabase’s lack of drivers in X language. Later on you’re dealing with stability and scalability issues you would never see with BattleTestedRDBMS.
The amount of capability a well designed Postgres instance can output is insane. I’ve seen a single vertically scaled Postgres instance compete with 100+ node Spark clusters on computations.
Zealotux|4 years ago
davidkell|4 years ago
humbleMouse|4 years ago
throwaway81523|4 years ago
Basics of cryptography: there are many dumb errors to avoid.
Antirez's general advice about "10x programmers" is good: http://antirez.com/news/112
Thorough (not just basic) knowledge of SQL, if you don't count that as a language. The sqlite.org "technical and design documents" about sqlite's virtual machine and its query planner are well worth reading, and apply to other databases as well. ORM's are less important than SQL, and are usually language specific as someone mentioned.
Reasonable clue about socket programming, even if you're doing everything with libraries that wrap the details.
Comfort using debugging and profiling tools.
Lots of other stuff, I'm sure.
comprev|4 years ago
nogbit|4 years ago
What data is coming into your system and it’s variety, velocity and volume.
Do you really need NoSQL, probably not.
Do you really need that ORM and all the schema, migrations and ops to go with it, known the pros and cons.
Are your boundaries defined well? Networking, firewalls etc? Are or do they need to be identity aware?
Are you logging what you need to log, where you need to log it and do the right people have access to it? Maybe metrics are really what you need.
What’s the dev story like? Can I run things locally or easily without spending days recreating an environment? IAC is one thing, but debugging some Python locally vs deploying and print statements sucks. Have a good readme and leave the repo better than you found it.
Tackle the hard problems first, then create reproducible developer story, then hand it off to someone Jr. so they can do the grunt work and you can help them out in a jiff.
CI/CD, incrementally improve it over time and don’t spend time boiling the ocean here. A simple bash script to deploy may suffice for an SRE to take it to the next step as IAC or to drop it into some CI tool.
Apply the practice of least privilege from the very start.
KISS, if what you are building is too confusing, it’s because you need to spend more time writing about it vs actually writing it.
tekstar|4 years ago
If you understand the system a layer of abstraction or two below the layer you work in, you will be able to debug deeper. Learn system calls, Various ways how to examine processes.
I learnt a lot of this back in the day by completing war games on a site called digital evolution (dievo). Those are antiquated now but still a really fun way to learn it.
pdpi|4 years ago
tifadg1|4 years ago
cudgy|4 years ago
Diggsey|4 years ago
- Fault tolerance. - Backwards (or forwards) compatibility. - Scalability. - Testability. - Everything around state (backup/restore, migration strategies, data integrity, etc.)
Most other things are a one-time cost. These things are an ongoing burden to consider, but if you forget to consider them it can be devastating.
Also remember: any time you give a (internal or external) customer programmatic access to something, that is an API, and APIs have huge costs to maintain. That includes when you dump your database into "data lake" for internal reporting...
ianpurton|4 years ago
- Database Migrations
- Kubernetes
- Basic RPC and code generation i.e. gRPC, OpenAPI and GraphQL.
- Realtime Concepts, i.e. Kafka, MQTT
- DevSecOps
- Builds. i.e. make files.
- Jobs, i.e. cron or batch and job workflows.
- Offsite incremental DB backups and restore.
- Infrastructure as Code i.e. Pulumi.
comprev|4 years ago
- Basics of server/runtime environment security (RBAC, least privileges, common threats, etc.)
agentultra|4 years ago
- Authentication: OAuth2 is probably the most widely used
- Authorization: RBAC
- Some rudimentary statistics: know how to read metrics, write metrics, etc
- Learn one RDBMS inside and out. Other database systems have their place but you’ll almost always encounter a Postgres, MySQL, MSSQL. Learn how to read EXPLAIN output, cursor based pagination, and indices.
chris_j|4 years ago
Understand how to load test your system and to reason about its behaviour under load and its failure modes when you push it too hard. It's one thing to be able to build a system and functionally test it such that you're confident that it behaves correctly when you send one request at a time. It's another thing to let thousands or millions of real users hit it for real in production and to have confidence that you are giving them all a good experience.
gilfoyle|4 years ago
https://martinfowler.com/books/eaa.html
bbkane|4 years ago
stueynz|4 years ago
Are you using the PII data for purposes other than it was originally collected?
Can you synthesise a good enough set of test data so you don’t have to anonymise production data? Hint: you can’t sufficiently anonymise production data and still have it be useful
yewenjie|4 years ago
Sonata|4 years ago
- The correct semantics for each HTTP method
- What different status codes indicate
- Common headers, particularly around caching
- HTTP 1.1 vs HTTP 2
- Common authentication protocols - OAuth 2.0, JWTs, etc.
bovermyer|4 years ago
- the OSI model, DNS, TCP/UDP, TLS, and networking in general
- CPU flame graphs and other low-level performance/debugging tools
- the Knightmare devops story
- anger management
rasikjain|4 years ago
1) RDBMS, NoSQL Concepts
2) Writing Queries and Joins
3) Connecting to Database native and ORM
4) HTTP Verbs like POST, GET, DELTE, PUT etc
5) Restful API and GraphQL Concepts
6) Session State, Application State, Caching and Safe Error Handling
7) Distributed Systems, SOA, Async Functions (i.e file handling)
8) Design Patterns, OOPs concepts (Abstraction, Interfaces etc)
9) Authentication, Authorization, Cryptography
10) Configuration, Minimum Privileges (e.g dbrole, server account etc)
eatonphil|4 years ago
corobo|4 years ago
No better way to know how to secure your code than the mindset of "Ok how would I break into this" :)
perelin|4 years ago
https://roadmap.sh/backend
aszen|4 years ago
These days most backend engineers also tend to manage data sources so understanding them is also a plus
koolba|4 years ago
etaham|4 years ago
aristofun|4 years ago
The sooner you get this attitude the more comfortable youll be with your skills development.
slifin|4 years ago
So that you can perform tests without firing the side effects
tompazourek|4 years ago
eterps|4 years ago
dudul|4 years ago
onei|4 years ago
Aeolun|4 years ago
Dependency inversion/injection
throwaway019254|4 years ago
Oleg2tor|4 years ago