I spent a lot of years as a PM and PgM, and the general gist of this article seems to be that if you have better PMs your project outcomes will be better. I know HN loves to talk about how much the PMs they've reported to suck, but having worked with many clients on dozens of projects, it's rarely that simple. While bad PMs are definitely way too common, so are bad development teams. And remember that PMs are usually getting shit on from two directions at once (development teams and management teams) with often very little authority to actually fix things. It's usually bad management practices and attitudes that sink projects and make life hell for everyone, and the cause of that is usually lack of understanding and empathy for how software development works.
The greatest PM in the world can't help a project if the executive sponsors keep changing scope and requirements or making ridiculous promises to clients to increase sales (and I've seen good PMs quit over this). We don't just need better project management training, we need better management training in general for anyone whose job is dependent on software. Also, companies need to actually be willing to pay for good PMs instead of pulling the cheapest contractor they can find and acting surprised when regurgitating the PMI handbook doesn't actually work.
We just onboarded a new PM with loads of experience. I was very skeptical at first, but within 3 weeks I was being kept honest by this new manager on our daily calls. I didn't realize that we've never had a "good" one and how amazing it might be.
I feel like the process for building your software is more important than the software itself. I.e. How can you worry about building cars when the factory that produces them is burning down, or experiencing a work stoppage because no one ever thought to develop a maintenance schedule for a particularly tricky piece of equipment?
The more I look at this whole enterprise of software engineering as a virtual factory floor, the easier it gets to draw analogies between developers and project managers regarding how we should manage work.
> The greatest PM in the world can't help a project if the executive sponsors keep changing scope and requirements or making ridiculous promises to clients to increase sales
It's turtles all the way down. So, let's think about executive sponsors here. Why do they change scope and requirements? Why do the act the way they do?
Yes, there are executives who's career trajectory is defined largely by luck rather then acumen, let alone being knowledgeable or empathic to those in their service, or the stakeholders whom they cater to. But that's not the entire story.
Businesses, whether a small 2 man outfit or billion dollar enterprise, do not operate in a void. Their behaviour is very much driven by changes in their environment. Whether that's the markets they are active in, customers they target, labour markets they tap into, competitors innovating and bringing new products and services to market, and public governance issuing a changing legal frameworks.
Changes in scope and requirements are par for the course. Executive sponsors perceive those as investments and try to make risk-benefit trade off's assessing the information they have at hand. This is where uncertainty seeps into the decision making process. Even with clearly quantified information, it's still hard to predict the future. How will markets evolve? What do customers want? What will the competition do one or two years out?
Strategic thinking and planning is hard because of uncertainty. In a military context, this is a clear given for leadership. Helmuth Von Moltke the Elder (1800-1891) famously said:
> The tactical result of an engagement forms the base for new strategic decisions because victory or defeat in a battle changes the situation to such a degree that no human acumen is able to see beyond the first battle. In this sense one should understand Napoleon's saying: "I have never had a plan of operations." Therefore no plan of operations extends with any certainty beyond the first contact with the main hostile force. [1]
> we need better management training in general for anyone whose job is dependent on software
For sure, leadership training is an established need. And it's also a thing that is catered to. There are entire libraries, fields of study, management courses and so on dedicated to leadership training. There's no lack of options out there. That's not the issue.
More interestingly, you have to first question how executives ended up in a leadership position in the first place. For instance, many leaders don't necessarily became leaders because of innate qualities that show a due sense of what is understood as "being a natural leader", but because they were born into a context that handed them the opportunities that put them on that trajectory. Do note how I say "many". The above isn't a catch all either. How individuals end up in a leadership or management role is really a matter of many complex factors and happen stance.
And so, having a social studies background, I always tend to be a bit weary when I see the "we need..." argument because (a) the "we" is never specified, (b) the "we need" negates complex contextual factors and (c) it reduces the argument into a single problem that can be resolved in general terms with a single pragmatic solution.
And so that brings me to this point:
> usually lack of understanding and empathy for how software development works
I feel this goes both ways. Trying to understand the reasoning or the why behind shifting business strategies, and realizing that these are driven by fallible human beings who make mistakes based on incomplete information goes a long way. Working towards improving software development practices is a a great goal, but good will never become perfect. And trying too hard to reach perfection might even backfire.
A lack of understanding and empathy of how software development works could be forgiven. A lack of understanding and empathy of how human beings feel and think - whether it's developers, project managers or executives - can't. It's the latter which makes all the difference at the end of the day.
If management don't understand the projects, and they generally don't, what are they really managing? Oh yeah, get the pay but use position as shield and sword.. You'll see the few exceptions are acting more like a PM, on and off.
He seems to be pushing his book and possibly at the end his school:
"Years ago I wrote a book – IT's All about the People: Technology Management That Overcomes Disaffected People, Stupid Processes and Deranged Corporate Cultures – that focused on the human element in technology. The premise is as correct today as it was then."
Projects are hard and deal with the unexpected. If you've never done anything like whatever you are trying to do before, you will encounter delays. These are really not failures -- in fact a lot of projects are "finished" with many additional "add on" pieces and enhancements scoped for future work.
Tellingly, companies that do the same thing over and over again (video game companies come to mind), are actually pretty good at delivering on time and on budget. With a stable and experienced workforce, the right corporate culture and reusable work breakdown structures, it's all possible.
Software projects are development projects, not constructions projects. They are similar to the engineer developing the blueprint for a new machine, not similar to the mechanic assembling it.
That's not entirely correct, given that most software projects tend to deliver new versions of something that already exists. Maintenance releases generally tend to be successful.
It's new projects that are hard and often have disastrous failures or cost overruns. I don't think software is unique in that regard. The Berlin Brandenburg Airport [1] and the new SF Bay Bridge [2] are cases in point. The latter included serious concerns about the integrity of the bolts that secure the East end of the suspension cables to its pier.
Maybe it becomes time to change that. Less innovation and more repetition could help here. I bet you, a company that picks a tool stack and develops several similar projects over several years will become better at it.
We could achieve such a setup by giving software a lifetime upfront and reimplement it after.
I've observed and participated in a reasonable number of projects, both IT and otherwise. In my experience there are two problems that seem to exit in every project that either failed or had to go on to life support:
1. The deliverables were not clearly defined and maintained.
How can you deliver something if you don't know what it is? In many cases there is either not a clearly defined and understood deliverable. This leads to every project officer having a different idea as to what they had to deliver leading to disagreements and conflicts and allowing external manipulation and scope creep.
2. People who have a vested interest in the successful outcome of the project are not directly involved in the delivery of that project.
How many times have you seen a project team employed to deliver "the new product", only to then have it handed over to the operational support team, who then have to try to work out what this monster is that they have been given and how they are every going to get it to work in the way it was originally specified. You see this a lot when you have a system replacement being designed and developed by an external team, then handed to the people who supported the old system.
Who best knows the task the old system was designed to complete, who knows the edge cases and peculiarities that system had to handle, and in most cases, who came up with the work arounds that kept the old system working.
If you are going to replace a system, and you are going to maintain your current team to support the new system, then get them working on the development of the new system. Not only will they be invaluable in identifying edge cases that the project architects may not be aware of, but as they are going to be the ones who will have to live with the new system, they will do their bests to make bloody sure that it's going to do what is's supposed to. Oh and make sure they are in a position to call out things that just won't work.
> People who have a vested interest in the successful outcome of the project are not directly involved in the delivery of that project.
I see that a lot at work. I can easily spot the birth of a troubled project just by looking if the people who'll be using it is represented or not.
In fact just last week I saved our customer a lot of money and pain by pushing on a team leader to get involved in their project to switch ERP solution. I was working on the integration between the new ERP and our program and noticed the users of our program were not involved at all, so leaned on the team lead to rustle her feathers. This team is small but performs an essential job for the company, if they can't do their job things grind to a halt.
After the team leader got involved she quickly discovered another department had gotten a change in which would be a complete showstopper for her team. Undoing this change after it had rolled out would have been non-trivial. Fortunately it was still early days in the project.
This is the most recent example, but yeah I see it all the time. The ones where the right people aren't involved invariably end up bad in one way or another.
I'm going to go out on a limb and say that smaller project are a possible solution. Do less and deliver something.
I've just gotten off two, multi-year projects, the first crashed and burned due to a mismatch between business process, vendors and the skills in the organization to understand what it does and translate that into requirements.
From the development side, we made a lot of stuff. Most of it was junk.
The second project arrived at a point underdone. Unable to replace the existing production solution because of a lack of a burning deck. I'd put this down as a failure as well. Mostly on the strategic leadership side.
In the meta sense these projects weren't that bad. People got money for doing them, and that money was spent in the economy.
Maybe Amazon is onto something with their two pizza teams, and 7 page proposals.
Smaller teams, and product management should be done by the development team. This will only work with at least some experienced developers on board who have shipped something before and had to deal with the results of that in production as well. For dedicated product managers the risk is high that the person does not know enough about the product or development. This is especially true in organizations where product management is an "easier" career path compared to development.
> [Research shows that] Only 58% of organizations fully understand the value of Project Management
> Why do only 58% of companies value project management?
Well, that seems to be taking the research totally out of context in the first paragraph.
And then just rambles on about having great people, and having them really well trained in project management. Wow, who would have thought being good at project management and making people better at project management was the solution to getting better at project management!?
"Is scope creep the problem? No, thats just a symptom of people again, and training. Did I mention I have a book?"
Maybe I'm just cynical here, but there doesn't seem like too much insight here.
From my experience it all comes down to budget. If IT is a cost, then the ideal thing to do is to deliver while minimizing that cost (duh!). When you do that you get solutions that are the quickest and cheapest thing at that moment.
This actually has further impact on other projects because everything is done badly.
A key elephant in the room in this article: if it's just about hiring the right people, why is that so hard? Because most companies treat technology as a cost center, not a profit center.
Even for companies where that's the case, treating technology like a cost center is like treating legal as a cost center -- existential company killing risk goes way up if you don't pay for the real deal. And for companies where it isn't, you'll just flat out waste your money. Pay for a Yugo when you need a big rig and you'll get exactly what you paid for, which won't be what you need.
It's true that leadership is the key. Projects are successfully lead, not managed.
This means leading those above too: getting more resources, communicating and managing expectations
The nature of projects is that they're often one-offs or something that the company hasn't done before. So unless you're being very generous with time, scope and quality estimates you're likely to be way off because of optimism bias
“I wrote a book on how PM is hard. It starts by hiring the right people. Of course, I wrote a book detailing how all the people involved at scale keep screwing it up. Buy my book.”
This a circular advertisement for a book.
I am guessing it’s more to with how most organize socially for human emotional reasons first and tacitly don’t give a shit about ephemeral nonsense except to play “Society: The Board Game”.
Maybe you should stop thinking your stupid L33T coding interviews will get you the best engineers. And actually interview your engineers for stuff that matters.
Like how they achieve redundancy, resilience, and fidelity in their software development.
And also, get some managers with half a brain, to understand that changes to the requirements can affect development efforts.
Tech projects should be easy to manage buy it fails due to,
1. Because of time sheet. It's either not tracked or questioned or sometimes it's questioned too much.
2. PM is not technical and believes the story being told.
3. PM calls for a meeting and asks for status but doesn't go to each people to collect information who is delaying or is he waiting for someone else's input. Flow is cut somewhere.
4. With sprints, too many managers but all sprints are handled by few Dev's, DBAs and testers.
5. Project changed its course into a discovery journey and everybody is too focused on the side track.
Projects fail because you have misaligned incentives. The CEO gets a pat on the back when he announces a Technology Modernization project, The Executive team gets a pat on the back when they negotiate 20% off the Modernization Project with IBM. The IBM salesman gets his commission as soon as the deal is signed.
Then when the project fails everyone blames someone else. If you want the project to succeed tell all those people that they will get their reward at the end of a successful project not before it has even started.
My question is, I wonder how Apple does it. Or May be Netflix. I think that is the only two in FAANG that is any good in product / project at a large scale.
Apple brute-forces it. They announce or decide to announce a release and do what's needed to meet that date. Netflix relies on people being driven to do the right thing with the consequence that those that don't deliver will be shown the door.
Reading this article calls to mind von Tiesenhausen's Law of Program Management: "To get an accurate estimate of final program requirements, multiply the initial time estimates by pi, and slide the decimal point on the cost estimates one place to the right."
Poor article in my opinion. The problem described in the article is very true. But the solution, “focus on people”, is mostly tautologic and doesn’t explain at all what makes some projects successful and not others. I’m disappointed.
Something that always seems to get missed is successful projects are linked to motivation. If everyone in the project is motivated to make it a success it's more likely to be.
Use an old technology none of the programmers are interested in and they go find other jobs or not work very hard as they put there efforts into their more interesting side projects..
As an example, projects linked to fulfilling legal requirements normally succeed because the motivation is strong - failure = fine/prison time.
Most projects I've seen failure where senior managnent doesn't want the project anymore but need a way of getting rid of it while saving face. Classic way is to pass the project onto another department...
My pet solution is to use traditional MVC frameworks like Django, Laravel, use Bootstrap stock themes for style, and sprinkle in specific JS wdigets as needed. This will (IME) make day to day work much less chaotic. The chaos of changing business requirements will be amplified when we reimplement API and React for every page and errant JS breaking even scroll.
If I can dream, non-tech guys can only mention high-level goals based on data than interface. Instead of the mandate "in one day, reimplement snazzy chart from slick app...", tell "show top 5 contributors" and tech guys can provide an economical solution first and iterate on it later.
[+] [-] mdorazio|5 years ago|reply
The greatest PM in the world can't help a project if the executive sponsors keep changing scope and requirements or making ridiculous promises to clients to increase sales (and I've seen good PMs quit over this). We don't just need better project management training, we need better management training in general for anyone whose job is dependent on software. Also, companies need to actually be willing to pay for good PMs instead of pulling the cheapest contractor they can find and acting surprised when regurgitating the PMI handbook doesn't actually work.
[+] [-] bob1029|5 years ago|reply
I feel like the process for building your software is more important than the software itself. I.e. How can you worry about building cars when the factory that produces them is burning down, or experiencing a work stoppage because no one ever thought to develop a maintenance schedule for a particularly tricky piece of equipment?
The more I look at this whole enterprise of software engineering as a virtual factory floor, the easier it gets to draw analogies between developers and project managers regarding how we should manage work.
[+] [-] unknown|5 years ago|reply
[deleted]
[+] [-] CaptArmchair|5 years ago|reply
It's turtles all the way down. So, let's think about executive sponsors here. Why do they change scope and requirements? Why do the act the way they do?
Yes, there are executives who's career trajectory is defined largely by luck rather then acumen, let alone being knowledgeable or empathic to those in their service, or the stakeholders whom they cater to. But that's not the entire story.
Businesses, whether a small 2 man outfit or billion dollar enterprise, do not operate in a void. Their behaviour is very much driven by changes in their environment. Whether that's the markets they are active in, customers they target, labour markets they tap into, competitors innovating and bringing new products and services to market, and public governance issuing a changing legal frameworks.
Changes in scope and requirements are par for the course. Executive sponsors perceive those as investments and try to make risk-benefit trade off's assessing the information they have at hand. This is where uncertainty seeps into the decision making process. Even with clearly quantified information, it's still hard to predict the future. How will markets evolve? What do customers want? What will the competition do one or two years out?
Strategic thinking and planning is hard because of uncertainty. In a military context, this is a clear given for leadership. Helmuth Von Moltke the Elder (1800-1891) famously said:
> The tactical result of an engagement forms the base for new strategic decisions because victory or defeat in a battle changes the situation to such a degree that no human acumen is able to see beyond the first battle. In this sense one should understand Napoleon's saying: "I have never had a plan of operations." Therefore no plan of operations extends with any certainty beyond the first contact with the main hostile force. [1]
[1] https://en.wikiquote.org/wiki/Helmuth_von_Moltke_the_Elder
> we need better management training in general for anyone whose job is dependent on software
For sure, leadership training is an established need. And it's also a thing that is catered to. There are entire libraries, fields of study, management courses and so on dedicated to leadership training. There's no lack of options out there. That's not the issue.
More interestingly, you have to first question how executives ended up in a leadership position in the first place. For instance, many leaders don't necessarily became leaders because of innate qualities that show a due sense of what is understood as "being a natural leader", but because they were born into a context that handed them the opportunities that put them on that trajectory. Do note how I say "many". The above isn't a catch all either. How individuals end up in a leadership or management role is really a matter of many complex factors and happen stance.
And so, having a social studies background, I always tend to be a bit weary when I see the "we need..." argument because (a) the "we" is never specified, (b) the "we need" negates complex contextual factors and (c) it reduces the argument into a single problem that can be resolved in general terms with a single pragmatic solution.
And so that brings me to this point:
> usually lack of understanding and empathy for how software development works
I feel this goes both ways. Trying to understand the reasoning or the why behind shifting business strategies, and realizing that these are driven by fallible human beings who make mistakes based on incomplete information goes a long way. Working towards improving software development practices is a a great goal, but good will never become perfect. And trying too hard to reach perfection might even backfire.
A lack of understanding and empathy of how software development works could be forgiven. A lack of understanding and empathy of how human beings feel and think - whether it's developers, project managers or executives - can't. It's the latter which makes all the difference at the end of the day.
[+] [-] _y5hn|5 years ago|reply
[+] [-] hungryforcodes|5 years ago|reply
"Years ago I wrote a book – IT's All about the People: Technology Management That Overcomes Disaffected People, Stupid Processes and Deranged Corporate Cultures – that focused on the human element in technology. The premise is as correct today as it was then."
Projects are hard and deal with the unexpected. If you've never done anything like whatever you are trying to do before, you will encounter delays. These are really not failures -- in fact a lot of projects are "finished" with many additional "add on" pieces and enhancements scoped for future work.
Tellingly, companies that do the same thing over and over again (video game companies come to mind), are actually pretty good at delivering on time and on budget. With a stable and experienced workforce, the right corporate culture and reusable work breakdown structures, it's all possible.
[+] [-] smarx007|5 years ago|reply
[+] [-] konschubert|5 years ago|reply
[+] [-] ardit33|5 years ago|reply
[+] [-] hodgesrm|5 years ago|reply
It's new projects that are hard and often have disastrous failures or cost overruns. I don't think software is unique in that regard. The Berlin Brandenburg Airport [1] and the new SF Bay Bridge [2] are cases in point. The latter included serious concerns about the integrity of the bolts that secure the East end of the suspension cables to its pier.
[1] https://en.wikipedia.org/wiki/Berlin_Brandenburg_Airport#Ove...
[2] https://www.wnyc.org/story/316201-brief-history-64-billion-b...
[+] [-] choeger|5 years ago|reply
We could achieve such a setup by giving software a lifetime upfront and reimplement it after.
[+] [-] dragonsky|5 years ago|reply
1. The deliverables were not clearly defined and maintained. How can you deliver something if you don't know what it is? In many cases there is either not a clearly defined and understood deliverable. This leads to every project officer having a different idea as to what they had to deliver leading to disagreements and conflicts and allowing external manipulation and scope creep.
2. People who have a vested interest in the successful outcome of the project are not directly involved in the delivery of that project.
How many times have you seen a project team employed to deliver "the new product", only to then have it handed over to the operational support team, who then have to try to work out what this monster is that they have been given and how they are every going to get it to work in the way it was originally specified. You see this a lot when you have a system replacement being designed and developed by an external team, then handed to the people who supported the old system.
Who best knows the task the old system was designed to complete, who knows the edge cases and peculiarities that system had to handle, and in most cases, who came up with the work arounds that kept the old system working. If you are going to replace a system, and you are going to maintain your current team to support the new system, then get them working on the development of the new system. Not only will they be invaluable in identifying edge cases that the project architects may not be aware of, but as they are going to be the ones who will have to live with the new system, they will do their bests to make bloody sure that it's going to do what is's supposed to. Oh and make sure they are in a position to call out things that just won't work.
[+] [-] magicalhippo|5 years ago|reply
I see that a lot at work. I can easily spot the birth of a troubled project just by looking if the people who'll be using it is represented or not.
In fact just last week I saved our customer a lot of money and pain by pushing on a team leader to get involved in their project to switch ERP solution. I was working on the integration between the new ERP and our program and noticed the users of our program were not involved at all, so leaned on the team lead to rustle her feathers. This team is small but performs an essential job for the company, if they can't do their job things grind to a halt.
After the team leader got involved she quickly discovered another department had gotten a change in which would be a complete showstopper for her team. Undoing this change after it had rolled out would have been non-trivial. Fortunately it was still early days in the project.
This is the most recent example, but yeah I see it all the time. The ones where the right people aren't involved invariably end up bad in one way or another.
[+] [-] gonzo41|5 years ago|reply
I've just gotten off two, multi-year projects, the first crashed and burned due to a mismatch between business process, vendors and the skills in the organization to understand what it does and translate that into requirements. From the development side, we made a lot of stuff. Most of it was junk.
The second project arrived at a point underdone. Unable to replace the existing production solution because of a lack of a burning deck. I'd put this down as a failure as well. Mostly on the strategic leadership side.
In the meta sense these projects weren't that bad. People got money for doing them, and that money was spent in the economy.
Maybe Amazon is onto something with their two pizza teams, and 7 page proposals.
[+] [-] mrich|5 years ago|reply
[+] [-] Closi|5 years ago|reply
> Why do only 58% of companies value project management?
Well, that seems to be taking the research totally out of context in the first paragraph.
And then just rambles on about having great people, and having them really well trained in project management. Wow, who would have thought being good at project management and making people better at project management was the solution to getting better at project management!?
"Is scope creep the problem? No, thats just a symptom of people again, and training. Did I mention I have a book?"
Maybe I'm just cynical here, but there doesn't seem like too much insight here.
[+] [-] eecks|5 years ago|reply
This actually has further impact on other projects because everything is done badly.
[+] [-] yowlingcat|5 years ago|reply
Even for companies where that's the case, treating technology like a cost center is like treating legal as a cost center -- existential company killing risk goes way up if you don't pay for the real deal. And for companies where it isn't, you'll just flat out waste your money. Pay for a Yugo when you need a big rig and you'll get exactly what you paid for, which won't be what you need.
[+] [-] jack_riminton|5 years ago|reply
This means leading those above too: getting more resources, communicating and managing expectations
The nature of projects is that they're often one-offs or something that the company hasn't done before. So unless you're being very generous with time, scope and quality estimates you're likely to be way off because of optimism bias
[+] [-] withPurpose9973|5 years ago|reply
This a circular advertisement for a book.
I am guessing it’s more to with how most organize socially for human emotional reasons first and tacitly don’t give a shit about ephemeral nonsense except to play “Society: The Board Game”.
[+] [-] blackrock|5 years ago|reply
Like how they achieve redundancy, resilience, and fidelity in their software development.
And also, get some managers with half a brain, to understand that changes to the requirements can affect development efforts.
[+] [-] bobbydreamer|5 years ago|reply
2. PM is not technical and believes the story being told.
3. PM calls for a meeting and asks for status but doesn't go to each people to collect information who is delaying or is he waiting for someone else's input. Flow is cut somewhere.
4. With sprints, too many managers but all sprints are handled by few Dev's, DBAs and testers.
5. Project changed its course into a discovery journey and everybody is too focused on the side track.
All this I have dealt.
[+] [-] bob33212|5 years ago|reply
Then when the project fails everyone blames someone else. If you want the project to succeed tell all those people that they will get their reward at the end of a successful project not before it has even started.
[+] [-] unknown|5 years ago|reply
[deleted]
[+] [-] ksec|5 years ago|reply
[+] [-] sg47|5 years ago|reply
[+] [-] tnr23|5 years ago|reply
[+] [-] hawktheslayer|5 years ago|reply
[+] [-] shuringai|5 years ago|reply
[+] [-] ngrilly|5 years ago|reply
[+] [-] docflabby|5 years ago|reply
Use an old technology none of the programmers are interested in and they go find other jobs or not work very hard as they put there efforts into their more interesting side projects..
As an example, projects linked to fulfilling legal requirements normally succeed because the motivation is strong - failure = fine/prison time.
Most projects I've seen failure where senior managnent doesn't want the project anymore but need a way of getting rid of it while saving face. Classic way is to pass the project onto another department...
[+] [-] aitchnyu|5 years ago|reply
If I can dream, non-tech guys can only mention high-level goals based on data than interface. Instead of the mandate "in one day, reimplement snazzy chart from slick app...", tell "show top 5 contributors" and tech guys can provide an economical solution first and iterate on it later.