I enjoyed "A philosophy of software design" by John K Ousterhout https://www.worldcat.org/title/philosophy-of-software-design... just be forewarned that at 180 pages, it's a lot more aspirational than instructive. It depends on what kind of organizational problem you're having: lack of experience, lack of interest, lack of discipline, lack of buy-in, or $other
The least complex software is the one never built. We often over-automate. Automation is brittle and inflexible and easily ends up lined with edge cases and other types o complexity.
Automation is great when it replaces a stable, well-working manual process.
The way to introduce automation is to first experiment with humans doing something manually until you have a great process. Then take the dumbest, most reliable part of that and turn it into a computer program that alerts a human if it encounters an edge case.
Try not to handle edge cases in the software, but try to remove them from the greater system in which they occur.
Automate by elimination, not by complexity.
Good books in this vein would be anything with Taiichi Ohno in the title, or, perhaps better, as the author.
Conversely, automation creates a clear and well defined workflow for completing a task. There is nothing to "automate" if there is not already a manual process to doing it. Sure, sometimes the automation you introduce can be an over-engineered solution that's more difficult to maintain (or just not work) compared to a human-done manual process. To that point I think a valid solution would be to make the process itself more automation friendly, if possible given your circumstances.
Mine automation works for years without an issue. Flaky automation is meaningless.
> that alerts a human if it encounters an edge case.
No. It shouldn't alert anybody almost never if it's good. When you have 300 automation routines, even if it alerts once a year you will have some alert every day. So it needs to alert lower then that, when the world falls out or something...
I feel like a broken record for the amount of comments I've posted related to Deming and lean product development these past few days, but as Taiichi Ohno says, the biggest waste is overproduction.
A lot of the things we build into our software is complexity nobody really needs. One way of managing it is by never making it in the first place. This takes at least two things:
- Thorough discussion with the customer about what problem they really are trying to solve. They will come to you with what they think is the solution. That way leads unnecessary complexity.
- Rough economical analysis of choices. Often we do things because that's what one ought to do. Make a quick napkin calculations. A lot of the things one ought to do are not actually necessary.
There are way too many books to recommend here, but perhaps some of the most relevant are
- Deming's The New Economics, and Out of the Crisis,
- Reinertsen's Principles of Product Development Flow,
Having moved from external consult to in house dev this is so true. Most of our production systems are just spreadsheets that get emailed on schedule or trigger.
We have no web stack, no authentication required. No complex UI or UX.
It's so simple, our production manager and I sat down and came up with the system. It works very well and took a few weeks to get up and running. It would have taken months to solve the same problems with a typical web based interactive system. It's also less work and upskilling for users.
Unfortunately "not implementing a feature" is not my decision most of the time.
Most of the time we need to convince some customer that we can implement what he needs because even though everything is there "they are super special".
If you start with saying no - you won't get a sale, our sales of course try to explain that we are quite successful SaaS solution so they probably could work in way we propose and won't have to pay for custom development.
Mind you that it most of the time is not purely technical but often it is checking if they can get what they want so to say "checking the waters" if we are responsible vendor.
Sometimes it is just asserting dominance by asshole manager of a big company to see how far will company bend over to win the contract.
Good part is most of the time we simply get paid for "custom development" but the downside is when 1 year or 2 later such customer goes away this code stays and is not that useful for other customers.
So even if those customers pay quite a lot for custom dev, they externalize cost of software maintenance to SaaS vendor. Because I don't believe anyone is factoring that in the price even if customer pays monthly fees for 2 years, if he goes away we are left with dead code and even removing it costs money as you still have to do regression testing.
I would like to add “About Face : on interaction design” for a solid grounding/ guide on how to manage user expectation, project re-iteration, working within scope, and working with interfaces. Knowing what stance your app has, already makes a world of difference.
Interesting recommendations u/kqr , don’t know the last two.
has anyone also included a post deploy trim phase ? even through long discussions, the client's business might be affected by the new tool in ways that may make him reevaluate workflows or distribution of work, which mean potentially reshaping the code and maybe removing more stuff.
“Software Engineering at Google” has been eye opening for me. It hits what you are looking for in its first chapter and then goes into details of different aspects in later chapters.
1. Figure out the difference between your inherent and accidental complexity. The Cynefin framework can be a useful way of thinking about what types of complexity you have.
2. Attempt to reduce (or remove) the complexity at root before assuming you must model it. "The biggest tragedy in business is doing well that which should not be done at all." For example, we had an important user still on a legacy browser because they couldn't afford to upgrade their old laptop. Instead of adding a complicated legacy layer, we bought them a new laptop.
3. Think of the whole system, rather than just shifting the burden somewhere else within the system. This is why microservice architectures so often fail - engineers think they've reduced complexity because each service is so simple. Yet all they've done is move the complexity into the operational layer, where it's harder to deal with. Gerald Weinberg is a fun writer in this space.
4. In many organisations, a lot of people's salaries depend on their ability to add complexity, especially to processes but also to code. If you are working within such an organisation, you will have a Sysiphian task until you solve the organisational problem. Whether you can solve this problem depends on how much influence you have. This is why "skunkworks" projects so often work until they are re-integrated into the mainstream.
5. Once you've sorted out the organisational complexity, separated the inherent and accidental complexity, you'll be most of the way there. Keep reviewing and challenging everything. Complexity is a weed that just seems to grow unless you constantly tend to the garden.
> For example, we had an important user still on a legacy browser because they couldn't afford to upgrade their old laptop. Instead of adding a complicated legacy layer, we bought them a new laptop.
I'm curious about the specifics regarding the combination of "important user" and "couldn't afford to upgrade".
Was the user's importance social rather than financial? Were the hardware requirements particularly hefty?
This might seem more basic/beginner level but I really have enjoyed working through How To Design Programs. [1] It teaches you how to think about data, types, abstraction, and creating mini languages/sets of functions for robust, adjustable, and maintainable code. It's helped me immensely as I've started to build much more complicated stuff. It's amazing how you can follow the design recipies, break problems into parts, plug functions together, everything just works. Applying these principles in a typed language (TypeScript) has massively improved my ability to much more complex stuff that is also more safe and maintainable.
Peter Naur's "Programming as Theory Building" is a classic essay, and absolutely worth reading. This transcription [1] (with some extra remarks at the end by the transcriptor?) is pretty accessible.
It motivates small batch sizes, decision making at ground level, short cycle times, all the mechanics behind kanban, scrum, XP, without ever using any of their jargon.
And it moves up into economics by explaining e.g. cost of delay as a metric guiding useful decision making.
The book eatablishes a wonderful shared understanding of the principles underlying lean and agile, why they work and how to make them work, shared understanding between engineering and management.
I really like that tagline, "How to avoid programming yourself into a corner." Sometimes I've made stuff that I've hacked and hacked together until it barely works and it feels like you're just straining harder and harder to get it workable and then there's this sinking feeling of, "oh dear, I can never touch or change this or everything will crumble into an irreparable mess." Learning the SICP/HTDP way of creating a "language" to handle your problem with changed everything for me. It helps you to clearly lay out and work with the particular kinds of operations that your problem (and its variations) require.
As Hal Abelson explains in this podcast: [1]
> So you really take the attitude that, boy if I’m writing something complex, I need to think about what I’m doing as if I’m writing a language. Or just say that a little less onerously. When I think about the operations I want to produce in any kind of program I’m writing, I don’t want to think about only that particular problem I’m doing right then. I want to carve out a space that’s broad enough that I could be doing that particular problem and lots of variations and lots of similar ones.
This paper revolutionized my perspective on software engineering and ultimately how our company does business.
If you want to skip to the good part, Chapter 9 is where you want to be.
We were able to create a hybrid FRP system that uses SQLite & application-defined functions as the sandbox within which all of this hypothetical functional/relational business occurs. This is in production for several of our customers right now.
For Software specific issues i highly recommend David Parnas' collected papers in the book Software Fundamentals. He and his colleagues defined much of what is mainstream in today's Software Architecture/Organization practice.
Am I doing something wrong if I'm unable to find that Parnas collection as a digital book? I've read some of his papers and they were very influential on me. Would love the book but I rarely have the energy to bring a print copy of books.
Michael Feather's book on legacy code contains a lot of advice that ought to be obvious, and yet every time I'm handed a scary piece of complex legacy software, I leaf through Feather's book and I almost always find something useful.
[+] [-] mdaniel|4 years ago|reply
[+] [-] urthor|4 years ago|reply
Although its ideas are heavily couched in OO terminology, it's changed my ideas about interfaces and exposure to complexity utterly.
Not comprehensive, but every programmer should absolutely read it (it's a tiny book after all).
[+] [-] Mawr|4 years ago|reply
[+] [-] vendiddy|4 years ago|reply
It's refreshing that they take a first principles approach in managing complexity.
[+] [-] unknown|4 years ago|reply
[deleted]
[+] [-] kqr|4 years ago|reply
Automation is great when it replaces a stable, well-working manual process.
The way to introduce automation is to first experiment with humans doing something manually until you have a great process. Then take the dumbest, most reliable part of that and turn it into a computer program that alerts a human if it encounters an edge case.
Try not to handle edge cases in the software, but try to remove them from the greater system in which they occur.
Automate by elimination, not by complexity.
Good books in this vein would be anything with Taiichi Ohno in the title, or, perhaps better, as the author.
[+] [-] extheat|4 years ago|reply
[+] [-] lalwanivikas|4 years ago|reply
[+] [-] CyberDildonics|4 years ago|reply
[+] [-] majkinetor|4 years ago|reply
Wait what ?
Mine automation works for years without an issue. Flaky automation is meaningless.
> that alerts a human if it encounters an edge case.
No. It shouldn't alert anybody almost never if it's good. When you have 300 automation routines, even if it alerts once a year you will have some alert every day. So it needs to alert lower then that, when the world falls out or something...
[+] [-] kqr|4 years ago|reply
A lot of the things we build into our software is complexity nobody really needs. One way of managing it is by never making it in the first place. This takes at least two things:
- Thorough discussion with the customer about what problem they really are trying to solve. They will come to you with what they think is the solution. That way leads unnecessary complexity.
- Rough economical analysis of choices. Often we do things because that's what one ought to do. Make a quick napkin calculations. A lot of the things one ought to do are not actually necessary.
There are way too many books to recommend here, but perhaps some of the most relevant are
- Deming's The New Economics, and Out of the Crisis,
- Reinertsen's Principles of Product Development Flow,
- Ward's Lean Product and Process Development.
[+] [-] xupybd|4 years ago|reply
We have no web stack, no authentication required. No complex UI or UX.
It's so simple, our production manager and I sat down and came up with the system. It works very well and took a few weeks to get up and running. It would have taken months to solve the same problems with a typical web based interactive system. It's also less work and upskilling for users.
[+] [-] ozim|4 years ago|reply
Most of the time we need to convince some customer that we can implement what he needs because even though everything is there "they are super special".
If you start with saying no - you won't get a sale, our sales of course try to explain that we are quite successful SaaS solution so they probably could work in way we propose and won't have to pay for custom development.
Mind you that it most of the time is not purely technical but often it is checking if they can get what they want so to say "checking the waters" if we are responsible vendor.
Sometimes it is just asserting dominance by asshole manager of a big company to see how far will company bend over to win the contract.
Good part is most of the time we simply get paid for "custom development" but the downside is when 1 year or 2 later such customer goes away this code stays and is not that useful for other customers.
So even if those customers pay quite a lot for custom dev, they externalize cost of software maintenance to SaaS vendor. Because I don't believe anyone is factoring that in the price even if customer pays monthly fees for 2 years, if he goes away we are left with dead code and even removing it costs money as you still have to do regression testing.
[+] [-] prox|4 years ago|reply
Interesting recommendations u/kqr , don’t know the last two.
[+] [-] agumonkey|4 years ago|reply
[+] [-] zenkat|4 years ago|reply
From my perspective, the top "must-reads" are:
-- John K Ousterhout, A Philosophy of Software Design
-- Titus Winter (et al), Software Engineering at Google
-- Hanson and Sussman, Software Design for Flexibility
Other interesting titles that caught my eye:
-- Peter Naur, Programming as Theory Building
-- Scott Wlaschin, Domain Modeling Made Functional
-- Yehonathan Sharvit, Data-Oriented Programming: Unlearning objects (upcoming)
-- Nick Tune, Patterns, Principles, and Practises of Domain Driven Design
-- Robert L. Glass, Facts and Fallacies of Software Engineering
-- Donald Reinertsen, The Principles of Product development Flow
-- Eric Normand, Grokking Simplicity
[+] [-] jupp0r|4 years ago|reply
https://abseil.io/resources/swe-book
[+] [-] hutrdvnj|4 years ago|reply
https://www.seagnotes.com/index.html
[+] [-] blowski|4 years ago|reply
1. Figure out the difference between your inherent and accidental complexity. The Cynefin framework can be a useful way of thinking about what types of complexity you have.
2. Attempt to reduce (or remove) the complexity at root before assuming you must model it. "The biggest tragedy in business is doing well that which should not be done at all." For example, we had an important user still on a legacy browser because they couldn't afford to upgrade their old laptop. Instead of adding a complicated legacy layer, we bought them a new laptop.
3. Think of the whole system, rather than just shifting the burden somewhere else within the system. This is why microservice architectures so often fail - engineers think they've reduced complexity because each service is so simple. Yet all they've done is move the complexity into the operational layer, where it's harder to deal with. Gerald Weinberg is a fun writer in this space.
4. In many organisations, a lot of people's salaries depend on their ability to add complexity, especially to processes but also to code. If you are working within such an organisation, you will have a Sysiphian task until you solve the organisational problem. Whether you can solve this problem depends on how much influence you have. This is why "skunkworks" projects so often work until they are re-integrated into the mainstream.
5. Once you've sorted out the organisational complexity, separated the inherent and accidental complexity, you'll be most of the way there. Keep reviewing and challenging everything. Complexity is a weed that just seems to grow unless you constantly tend to the garden.
[+] [-] webmaven|4 years ago|reply
I'm curious about the specifics regarding the combination of "important user" and "couldn't afford to upgrade".
Was the user's importance social rather than financial? Were the hardware requirements particularly hefty?
[+] [-] elcapitan|4 years ago|reply
[+] [-] pluggles|4 years ago|reply
The entire focus of the book is reducing complexity.
[+] [-] adamddev1|4 years ago|reply
[1]: https://htdp.org/
[+] [-] unknown|4 years ago|reply
[deleted]
[+] [-] Twisol|4 years ago|reply
[1] https://pablo.rauzy.name/dev/naur1985programming.pdf
[+] [-] froh|4 years ago|reply
https://books.google.de/books/about/The_Principles_of_Produc...
It motivates small batch sizes, decision making at ground level, short cycle times, all the mechanics behind kanban, scrum, XP, without ever using any of their jargon.
And it moves up into economics by explaining e.g. cost of delay as a metric guiding useful decision making.
The book eatablishes a wonderful shared understanding of the principles underlying lean and agile, why they work and how to make them work, shared understanding between engineering and management.
[+] [-] bear8642|4 years ago|reply
[+] [-] myle|4 years ago|reply
[+] [-] MoritzWall|4 years ago|reply
Clean Architecture: A Craftsman’s Guide to Software Structure and Design https://www.oreilly.com/library/view/clean-architecture-a/97...
Organizational perspective:
An Elegant Puzzle: Systems of Engineering Management by Will Larson https://press.stripe.com/an-elegant-puzzle
[+] [-] ignoramous|4 years ago|reply
[+] [-] deltaonefour|4 years ago|reply
Organization is just a name for a group of methodologies. Ultimately what we want is the right way to organize programs for maximum flexibility.
[+] [-] adamddev1|4 years ago|reply
> So you really take the attitude that, boy if I’m writing something complex, I need to think about what I’m doing as if I’m writing a language. Or just say that a little less onerously. When I think about the operations I want to produce in any kind of program I’m writing, I don’t want to think about only that particular problem I’m doing right then. I want to carve out a space that’s broad enough that I could be doing that particular problem and lots of variations and lots of similar ones.
[1]: https://corecursive.com/039-hal-abelson-sicp
[+] [-] fred123|4 years ago|reply
[+] [-] bob1029|4 years ago|reply
If you want to skip to the good part, Chapter 9 is where you want to be.
We were able to create a hybrid FRP system that uses SQLite & application-defined functions as the sandbox within which all of this hypothetical functional/relational business occurs. This is in production for several of our customers right now.
[+] [-] rramadass|4 years ago|reply
I would add Frederick Brooks' No Silver Bullet alongside this for better understanding.
[+] [-] rramadass|4 years ago|reply
For Software specific issues i highly recommend David Parnas' collected papers in the book Software Fundamentals. He and his colleagues defined much of what is mainstream in today's Software Architecture/Organization practice.
Finally, the case studies given here are a great source of insights: https://aosabook.org/en/index.html
[+] [-] kqr|4 years ago|reply
[+] [-] yuppie_scum|4 years ago|reply
The Phoenix Project. The DevOps Handbook. The Google SRE Book. And “Kill It With Fire.”
[+] [-] ncfausti|4 years ago|reply
You want to minimize side-effects, preferring computations over them, and data over computations.
I’m still early into it, however, this already has been a helpful lens to view my code through.
https://www.manning.com/books/grokking-simplicity
[+] [-] triebelf|4 years ago|reply
https://www.amazon.com/dp/173210221X
[+] [-] kqr|4 years ago|reply
[+] [-] SideburnsOfDoom|4 years ago|reply
You can also google up a fair amount of related blog text, and video talks content.
[+] [-] zonovar|4 years ago|reply
https://www.amazon.co.uk/Pragmatic-Programmer-Andrew-Hunt/dp...