I find it somewhat amusing that this article doesn't pick up on the fact that it is furthering the root cause of this very issue in the act of basing it on the book Sapiens. A core problem with these mythological trends is that they reduce world views down to easy-to-communicate little stories. This is useful for propagating ideas from person to person, but it necessarily has to be pithy and avoid too much detail. What we aught to do is to notice this historical pattern, and to take a stand against it. We should try to not be too persuaded by the book, or methodology, of the month (Sapiens right now) but to try to go back to the hard, tough, classics. We should try to build a foundation based on actual psycological and sociological research, instead of being steered too much by the, understandably attractive, mimetic pulls of water cooler philosophy.
I used to think in terms of stories, but then I heard Tyler Cowen tell me a story about the dangers of stories, and now I don't think in terms of stories so much. (My story of personal growth.)
"they reduce world views down to easy-to-communicate little stories."
Yes, exactly. That's what you want. Make it too complicated and you lose people in the details. The content of the fiction you choose to operate under isn't as important as the collaborative power it unleashes.
I'm not sure that trying to fight human nature is going to be a winning strategy. Stories may be oversimplifying, but it is exactly that strategic oversimplification that is important to many endeavors.
Nice article! I'd like to a software methodology more supportive of what I'd call "Continuous refactoring" or a more "kaizen"-like mindset. "Agile" isn't always as agile as I'd like, but becomes rigid in peoples interpretation of the rules.
It's a constant fight to keep down the number of lines of code and complexity, in large projects. Once a code-base passes beyond a certain point it becomes difficult to see redundancies and patterns. Bloated code breeds bloated code.
If you on the other hand keep trimming and keep refactoring deep and wide as the code and requirements change you can often keep things manageable and understandable. "bad code" get's easier to spot, it's a little bit like the broken windows theory.
Scrum IMO and a lot of the agile methodologies, at least in their implementation, in my experience, tends to treat projects like brick-building and lacks a way to support and enforce constant re-architecturing and removal of code smells.
If code quality and refactoring is not supported by the actual methodology it can become an bureaucratic problem where product owners doesn't see the value and doesn't prioritize it (even though it leads to quicker development time and less bugs in the long run)
Unit testing is supposed to make refactoring easier, but can become an impediment to refactoring as well, as the cost of rewriting tests can be seen as too high.
We're currently experimenting with having one hour meeting each week where you can discuss things like that, and then dedicating some time to refactor, we'll see how it works out :)
That sounds like the 'healthy' team I talk about in the article. The problem with the 'continuous refactoring' (in my experience) is that the people that hold the purse strings cannot grasp the benefit, so cut this early.
The Agile methodology is to argue about the Agile methodology.
PMI style critical path, planning backwards from the release, is the only approach I’ve ever seen that was clear, concrete, actionable, high functioning.
I am ambivalent about "the difficulty lies not in defining [useful software methodologies], but in convincing others to follow [them]."
Currently, I see too much faith being put in methods of dubious utility. I see techniques, that may be effective when applied judiciously in some circumstances, being promoted to universal principles. In particular, I see the failure of waterfall being taken to mean that any attempt to think ahead is counterproductive.
On the other hand, ten years before the scope of this article, Barry Boehm took on the waterfall model with an iterative one that emphasized frequent reevaluation of where you are, where you are going, and what impediments you face [1]. The fact that this article begins with waterfall supports the notion that adoption is the problem. Perhaps Boehm's model was largely overlooked precisely because it was not dogmatic enough to promote zealotry?
Very true. I've found at every company that I've worked at that a "bad" process followed consistently by everyone always outperforms the "perfect" process followed haphazardly.
This reminds me of the recent threads about people using Excel instead of "real" software. At the end of the day, you are trying to add value. As long as you aren't adding technical debt or other risks, the how matters much less.
Methodologies are just virtue signals. Project management is just the self soothing that people do to pretend we’re not all just a bunch of howler monkeys fighting over whatever kibble is left over after the job creators take their share.
It's important for me to separate the principles behind these methodologies, from the industries that spawn up around them to sell guides and certifications and books and conference tickets.
Adhering to the ideas of clear goal-setting, team communication, autonomy, and iterative development have never had bad outcomes for me. But formalising them into a ruleset for any arbitrary team doesn't capture everything.
Okay. I'm not a zealot about these things, but I think the author underappreciates the nicer aspects of agile. My favorite book on agile looks at it from a mathematical / risk management / process model standpoint.
The concepts used in the book - batch size, uncertainty, feedback, control, queue size and latency - these really are useful things to think about when scheduling work.
That said, it seems the author is making a larger point than the efficacy of any individual methodology: Convincing people to co-operate within any reasonable methodology/framework has the larger effect than effects of different methodologies. That having a religion/philosophy is more important than which religion/philosophy in terms of national co-operation, etcetera.
He freely admits he has never read anything about any of this or made a serious study of it, or had to implement any of it. It is more of a perspective or list of observations from life in the trenches than a serious study or even a very informed position. The whole list of methodologies even mentioned in the article is just three: waterfall, hacking, agile.
I second your recommendation. Any product manager or engineering manager who hasn't read that book is probably working on the basis of many flawed assumptions and optimizing for the wrong metrics.
My approach is "Risk Driven Development" - find the most likely causes of failure for your project, and structure your methodology around reducing those.
The standard lean startup risk is building something the customer doesn't want, so it makes sense that the way to mitigate that is by getting customer feedback quickly.
But other projects can have substantially different risks - for example, one project I worked on involved building a pipeline to process a large amount of data every day. There was no real customer risk, the spec was pretty much "do this thing that's valuable on some data, but for 1,000x more." Instead, the major uncertainty was whether the libraries we considered would scale up. To mitigate this, we built small prototype projects with each library on production workloads, and found something like 7 out of 12 of them would work with minor adjustments, while 5 had to be discarded pretty much entirely. We then wrote the project with alternatives for those 5, or writing replacements for the functionality we needed from scratch.
As teams grow, coordination becomes a bigger risk. One way to reduce this is by having interfaces between modules, and enforcing those interfaces in code. My preferred method is static types and functions, but of course there are others like JSON APIs.
In all the years I've been doing this the only times I've felt things really gelled and were firing on all cylinders had little to do with methodology and everything to do with culture and motivation. When people felt cohesive, and they were all working towards the same goal, and that their personal priorities and the project priorities could align.
I do think that the 'agile' methodology focus on short achievable iterations and daily stand up meetings yielded good results _in that context_. Disconnected from a healthy organization or transplanted into a bad one, it just yields epic fail and falls apart.
Another thing that I saw work well was embedding product management, testing, and operations people into small per-project teams with developers, which I found reduced conflict from territoriality and reduced "throwing over the wall" syndrome.
In my soon-to-be 15-year career as a professional software engineer (and several more as an amateur) I've noticed that the common trait of the high functioning teams I have had the distinct pleasure to work with was always the same thing – a great leader. (In one case, there were two, working sort of in tandem.)
Trying to sum up these leaders' qualities is difficult, but if I try to distill it I think it comes down to primarily these two aspects:
1. Strong vision and ability to clearly communicate direction
2. Mutual trust
They wouldn't tell people what to do, and certainly not how to do it, but they'd always – every single day – point in the direction they wanted you to go and course correct where necessary. It wasn't like it was an all-hands meeting every morning either, it was more like gentle nudging or little reminders, but not unsolicited. It's difficult to explain frankly, but the result was that we as a team never felt unsure as to where we were heading.
The most important thing however certainly was the mutual trust. You could trust these leaders to have your backs, no matter what, and these people would trust you to get things done. That sense of trust created an immensely palpable sense of loyalty, you really did feel like your input and efforts mattered, and you always wanted to bring your a-game for the team. Again, difficult to explain, but you know when you're in it.
Not one of these teams worked like the other, in terms of methodology, yet things worked and they worked very well. We had issues, for sure, but I've also worked in dysfunctional teams and the same kind of issues in those teams were amplified by terrible leadership, leading to constantly worsening morale and in some unfortunate cases abject failure to deliver.
‘The test of a first-rate intelligence is the ability to hold two opposed ideas in mind at the same time and still retain the ability to function.’ F. Scott Fitzgerald
I know I am taking this out of context, but this is best thing I read on the internet since a while.
Companies really really like the words PMO and portfolio management. That's why Scaled Agile Framework(SAFe) is that nasty thing you see in that picture.
"People will benefit from SAFe, but not nearly as much as they would from a true agile approach"
IMHO all those methodologies that try to somehow shoehorn classical project management into new things are at least to some extent the acknowledgement that things aren't working as expected.
There is a tendency in Software Development, that terms lose their meaning the more they are used. I think it is partly due to software people not talking concrete, physical objects that much, but more about abstract ideas with more or less relevance to reality.
This can lead to interesting effects. For example, I have heard "DRY - Don't repeat yourself" most times I encountered it as a reasoning for obsessive refactorings, where functions where extracted from code without any regards to abstraction layer just because a few lines of code looked the same (or could be rewritten to look the same). But originally the term was coined to represent the principle of "Every piece of knowledge must have a single, unambiguous, authoritative representation within a system". "DRY" was however so catchy, that the slogan propagates without the explanation of the principle.
I think agile suffers a similar fate. Countless consultants, managers and engineers have rendered the word almost meaningless. Even with the Agile manifesto at hand, it is sometimes hard to free agile from the multiple layers of paint spread over it.
DevOps suffers the same fate, and while I haven't researched this, I assume that even waterfall is probably a gross misrepresentation of the original ideas.
Nice summary especially about the real intent of DRY and how people implement it. I think what it comes down to is that software development is very complex so any attempt to produce cookie cutter rules will fail eventually. OOP is a good idea, Agile is a good idea, DRY is a good idea but you still need the judgement to see how it applies to your specific situation.
I started doing this in the 90s and I feel that there is a growing trend to replace judgement with process and strict rules.
I think DRY is actually an axiomatic principle of good code. However, there are several axioms and they all conflict, so maximizing one at the expense of the others means you are writing bad code.
I've tried or been used to try a dozen or more methodologies over the years, more than I can easily name. Everything from SSADM and ITIL through RUP, various sorts of Agile... all methodologies are complete nonsense with the exception of Kepner-Tregoe.
I have seen parts of this in other training (six sigma, tqm, etc). But there is something very important in dedicating the time to think outside the most obvious immediate problem, which we often need to be reminded of by process.This was the link I found, but honestly the images in the Google search were better.
Lurker MBA student interested in PM roles here. Would be very interested to hear more about what you liked about Kepner-Tregoe, and how you learned it.
unified process is not complete non sense. It's the best I found so far and maybe the only methodology that covers projects longer than a few weeks and from start to finish.
The commercial IBM version (RUP: Rational Unified Process) with the associated tool (rationals TM) and consulting service is really horrible though.
The whole point of project management is using the right tool for the job. Sometimes that’s waterfall, sometimes it’s agile but most of the times it’s a mix.
A key point comes from the linked post entitled, "Why don't software methodologies work?" People are the greatest factor in a software project's success, not tools or methodology. Good tools and methodology are part of success, but they aren't sufficient for it.
If at any job you are using some methodology "exactly as it was written" (orthodox-style), without adapting it to the needs and vagaries of your company, your team/org/company is being run by someone who has probably never done the job themselves. I would advise polishing the resume and shopping it around.
If you use a software development methodology for something that isn't software development, you should again probably find a different company to work for, because things aren't going to end well there.
If every single project and task you work on uses only one methodology, the people in charge are either morons or full of shit. (Or you are actually a robot in a factory, in which case, hello, I appreciate your intelligence, please do not harm me)
Perhaps, but on the other hand too many managers wrongly think that their organizations are unique and special snowflakes. Sometimes plain vanilla is the best flavor, and many organizations would get better results by implementing a modern methodology exactly as written rather than wasting a bunch of time customizing it.
Didn't see it that time. Found it interesting this time.
Hacker News Guidelines What to Submit
On-Topic: Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: anything that gratifies one's intellectual curiosity.
[+] [-] tw1010|8 years ago|reply
[+] [-] danblick|8 years ago|reply
https://www.ted.com/talks/tyler_cowen_be_suspicious_of_stori...
(PS: my comment is actually just a joke from the talk, but in any case I really enjoyed and talk and would recommend it.)
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] kstenerud|8 years ago|reply
Yes, exactly. That's what you want. Make it too complicated and you lose people in the details. The content of the fiction you choose to operate under isn't as important as the collaborative power it unleashes.
[+] [-] _ea1k|8 years ago|reply
[+] [-] NickHolt|8 years ago|reply
[+] [-] mhomde|8 years ago|reply
It's a constant fight to keep down the number of lines of code and complexity, in large projects. Once a code-base passes beyond a certain point it becomes difficult to see redundancies and patterns. Bloated code breeds bloated code.
If you on the other hand keep trimming and keep refactoring deep and wide as the code and requirements change you can often keep things manageable and understandable. "bad code" get's easier to spot, it's a little bit like the broken windows theory.
Scrum IMO and a lot of the agile methodologies, at least in their implementation, in my experience, tends to treat projects like brick-building and lacks a way to support and enforce constant re-architecturing and removal of code smells.
If code quality and refactoring is not supported by the actual methodology it can become an bureaucratic problem where product owners doesn't see the value and doesn't prioritize it (even though it leads to quicker development time and less bugs in the long run)
Unit testing is supposed to make refactoring easier, but can become an impediment to refactoring as well, as the cost of rewriting tests can be seen as too high.
We're currently experimenting with having one hour meeting each week where you can discuss things like that, and then dedicating some time to refactor, we'll see how it works out :)
[+] [-] zwischenzug|8 years ago|reply
[+] [-] s73ver_|8 years ago|reply
I think a lot of this is because, if you don't enforce the rules rigidly, they tend to get cast by the wayside, especially by management.
[+] [-] nradov|8 years ago|reply
[+] [-] specialist|8 years ago|reply
PMI style critical path, planning backwards from the release, is the only approach I’ve ever seen that was clear, concrete, actionable, high functioning.
[+] [-] mannykannot|8 years ago|reply
Currently, I see too much faith being put in methods of dubious utility. I see techniques, that may be effective when applied judiciously in some circumstances, being promoted to universal principles. In particular, I see the failure of waterfall being taken to mean that any attempt to think ahead is counterproductive.
On the other hand, ten years before the scope of this article, Barry Boehm took on the waterfall model with an iterative one that emphasized frequent reevaluation of where you are, where you are going, and what impediments you face [1]. The fact that this article begins with waterfall supports the notion that adoption is the problem. Perhaps Boehm's model was largely overlooked precisely because it was not dogmatic enough to promote zealotry?
[1] https://en.wikipedia.org/wiki/Spiral_model
[+] [-] snarf21|8 years ago|reply
This reminds me of the recent threads about people using Excel instead of "real" software. At the end of the day, you are trying to add value. As long as you aren't adding technical debt or other risks, the how matters much less.
[+] [-] specialist|8 years ago|reply
[+] [-] ozmbie|8 years ago|reply
Adhering to the ideas of clear goal-setting, team communication, autonomy, and iterative development have never had bad outcomes for me. But formalising them into a ruleset for any arbitrary team doesn't capture everything.
[+] [-] danblick|8 years ago|reply
https://books.google.com/books/about/The_Principles_of_Produ...
The concepts used in the book - batch size, uncertainty, feedback, control, queue size and latency - these really are useful things to think about when scheduling work.
[+] [-] SubiculumCode|8 years ago|reply
[+] [-] patja|8 years ago|reply
[+] [-] nradov|8 years ago|reply
[+] [-] SatvikBeri|8 years ago|reply
The standard lean startup risk is building something the customer doesn't want, so it makes sense that the way to mitigate that is by getting customer feedback quickly.
But other projects can have substantially different risks - for example, one project I worked on involved building a pipeline to process a large amount of data every day. There was no real customer risk, the spec was pretty much "do this thing that's valuable on some data, but for 1,000x more." Instead, the major uncertainty was whether the libraries we considered would scale up. To mitigate this, we built small prototype projects with each library on production workloads, and found something like 7 out of 12 of them would work with minor adjustments, while 5 had to be discarded pretty much entirely. We then wrote the project with alternatives for those 5, or writing replacements for the functionality we needed from scratch.
As teams grow, coordination becomes a bigger risk. One way to reduce this is by having interfaces between modules, and enforcing those interfaces in code. My preferred method is static types and functions, but of course there are others like JSON APIs.
[+] [-] rileymat2|8 years ago|reply
[+] [-] cmrdporcupine|8 years ago|reply
I do think that the 'agile' methodology focus on short achievable iterations and daily stand up meetings yielded good results _in that context_. Disconnected from a healthy organization or transplanted into a bad one, it just yields epic fail and falls apart.
Another thing that I saw work well was embedding product management, testing, and operations people into small per-project teams with developers, which I found reduced conflict from territoriality and reduced "throwing over the wall" syndrome.
[+] [-] mstade|8 years ago|reply
Trying to sum up these leaders' qualities is difficult, but if I try to distill it I think it comes down to primarily these two aspects:
1. Strong vision and ability to clearly communicate direction 2. Mutual trust
They wouldn't tell people what to do, and certainly not how to do it, but they'd always – every single day – point in the direction they wanted you to go and course correct where necessary. It wasn't like it was an all-hands meeting every morning either, it was more like gentle nudging or little reminders, but not unsolicited. It's difficult to explain frankly, but the result was that we as a team never felt unsure as to where we were heading.
The most important thing however certainly was the mutual trust. You could trust these leaders to have your backs, no matter what, and these people would trust you to get things done. That sense of trust created an immensely palpable sense of loyalty, you really did feel like your input and efforts mattered, and you always wanted to bring your a-game for the team. Again, difficult to explain, but you know when you're in it.
Not one of these teams worked like the other, in terms of methodology, yet things worked and they worked very well. We had issues, for sure, but I've also worked in dysfunctional teams and the same kind of issues in those teams were amplified by terrible leadership, leading to constantly worsening morale and in some unfortunate cases abject failure to deliver.
[+] [-] agumonkey|8 years ago|reply
[+] [-] itsyogesh|8 years ago|reply
I know I am taking this out of context, but this is best thing I read on the internet since a while.
[+] [-] gtycomb|8 years ago|reply
'So I’m cool with it. Lean, Agile, Waterfall, whatever, the fact is we need some kind of common ideology to co-operate in large numbers. '
[+] [-] rjzzleep|8 years ago|reply
"People will benefit from SAFe, but not nearly as much as they would from a true agile approach"
https://twitter.com/martinfowler/status/440874515199844352
IMHO all those methodologies that try to somehow shoehorn classical project management into new things are at least to some extent the acknowledgement that things aren't working as expected.
[+] [-] nradov|8 years ago|reply
[+] [-] dovik|8 years ago|reply
Personally, I did not know this manifesto before.
[+] [-] auggierose|8 years ago|reply
[+] [-] wirrbel|8 years ago|reply
This can lead to interesting effects. For example, I have heard "DRY - Don't repeat yourself" most times I encountered it as a reasoning for obsessive refactorings, where functions where extracted from code without any regards to abstraction layer just because a few lines of code looked the same (or could be rewritten to look the same). But originally the term was coined to represent the principle of "Every piece of knowledge must have a single, unambiguous, authoritative representation within a system". "DRY" was however so catchy, that the slogan propagates without the explanation of the principle.
I think agile suffers a similar fate. Countless consultants, managers and engineers have rendered the word almost meaningless. Even with the Agile manifesto at hand, it is sometimes hard to free agile from the multiple layers of paint spread over it.
DevOps suffers the same fate, and while I haven't researched this, I assume that even waterfall is probably a gross misrepresentation of the original ideas.
[+] [-] maxxxxx|8 years ago|reply
I started doing this in the 90s and I feel that there is a growing trend to replace judgement with process and strict rules.
[+] [-] crdoconnor|8 years ago|reply
[+] [-] gaius|8 years ago|reply
[+] [-] kurthr|8 years ago|reply
http://begrip.be/kepner-tregoe-analysis/
[+] [-] shandor|8 years ago|reply
[+] [-] wbarber|8 years ago|reply
[+] [-] gm3dmo|8 years ago|reply
[+] [-] user5994461|8 years ago|reply
The commercial IBM version (RUP: Rational Unified Process) with the associated tool (rationals TM) and consulting service is really horrible though.
[+] [-] eksemplar|8 years ago|reply
[+] [-] zwischenzug|8 years ago|reply
[+] [-] bobthechef|8 years ago|reply
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] bpyne|8 years ago|reply
Definitely read that post.
[+] [-] peterwwillis|8 years ago|reply
If you use a software development methodology for something that isn't software development, you should again probably find a different company to work for, because things aren't going to end well there.
If every single project and task you work on uses only one methodology, the people in charge are either morons or full of shit. (Or you are actually a robot in a factory, in which case, hello, I appreciate your intelligence, please do not harm me)
[+] [-] nradov|8 years ago|reply
[+] [-] mparr4|8 years ago|reply
[+] [-] taspeotis|8 years ago|reply
[+] [-] kbutler|8 years ago|reply
Hacker News Guidelines What to Submit
On-Topic: Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: anything that gratifies one's intellectual curiosity.
[+] [-] dang|8 years ago|reply