Great article. When I worked at $OldJob, the leadership wanted an unsolved, research-grade problem solved in a few months. They demanded estimates, then refused to accept estimates beyond their timeline, then tried to hold people "accountable" for missing the estimates. Of course it was a mess, and of course we failed to hit our goals, and of course many many people burned out.
But I noticed that some people handled the situation fine. They stayed on management's good side, even though they were failing to deliver along with the rest of us. I will try to distill what I observed them do:
1) They did not fight on the estimates. If a manager forced them into a certain timeline, they registered their disagreement and just accepted the new timeline. I think they realized that fighting would just make the manager judge them less capable. (Pick your battles, eh?).
2) When the schedule slipped, they would communicate it in a way that made them seem more competent instead of less. The explanation usually had three parts: unforeseen events kept us from hitting the deadline, we accomplished some great things in the meantime, and here's why we're in great shape to hit the next deadline! For example: "When we made this schedule, we did not realize that AWS nodes were so unreliable! Despite this, our team has made incredible progress on implementing a fast method of storage and solid compression! We have reworked the schedule to reflect this new information, and we are already on track for the first milestone!"
It's possible that this trick just worked for the specific situation at $OldJob, but I really enjoyed learning it. They seemed to understand that certain explicit rules were not important, but that other unspoken rules needed to be followed. Are accurate estimates important? It depends on the situation! Sometimes wrong estimates can be more valuable than correct ones. Construction companies give wrong estimates all the time in order to win projects. Is staying on good terms with your boss important? Yes! Even if they are total shitbags, being adversarial won't help, only leaving will. These people demonstrated that there are often ways to fix a bad situation by breaking some explicit rules and carefully following implicit ones, and I wish I had the acuity to see these possibilities on my own.
> 2) When the schedule slipped, they would communicate it in a way that made them seem more competent instead of less.
In other words: Enable the bad estimates, then externalize accountability when they can't be achieved. In the past, I called these people "blameless go-getters" because they were always first to volunteer to take on work, but somehow it became everyone else's fault when they failed. If management is asleep at the wheel, it's a win-win situation for them.
You're exactly right that this method works in many companies. Like you said, it's all about understanding what the company actually values. When unrealistic schedules are forced on teams, the exact date might not be important. Instead, it might be leadership's dysfunctional way of emphasizing focus, urgency, and quick iteration. The savvy engineers and managers know how to put on a show that hits these key points while steadily delivering progress in the background.
Still, there's no escaping the fact that this is dysfunctional. More importantly, it doesn't have to be like this. It's eye-opening to move from a dysfunctional company like you described toward a company that values honest communication and understands engineering project management at the executive leadership level. When hiring someone out of a dysfunctional company like you described, it can take some time to break them of bad habits around schedule misdirection and estimation dishonesty.
This strikes me as an understandable but pathological response to managerialism. Managerialism being our current dominant business philosophy.
To see it in contrast, think for a moment about a well-run hospital. At a hospital, the people who make the key decisions about cases are medical professionals, not managers. If a surgery was supposed to take 4 hours but takes 12, well, that's how long the surgery took. Everybody recognizes that the 4-hour number was an estimate, and estimates are not commitments. The most important thing is patient health, not manager feelings. Managers help organize the work, but they do not control the work.
I would love to see software development become a true profession, where stroking manager egos by making them always feel correct and in control is not the most important thing.
This works because no-one is demanding estimates because they want to know the answer, they are doing it to apply pressure to get it done. The project manager is being subject to the same thing, and if her minions are savvy enough to provide a story that she can take to her management, and if she's smart enough to realize that's the most she can realistically hope for, a sort of equilibrium is achieved.
Stakeholder management is an important but often forgotten art. Keep them up to date on what they can expect. Don't fight them, just inform them. If a deadline is unrealistic, don't fight it, just let them know you cannot commit to that deadline with any certainty. Once it's clear you're not going to meet that deadline, let them know in advance.
- CYA above all. The chances of a significant project succeeding in such an environment are pretty slim. Prepare your umbrella from the get go for when the inevitably will hit the fan. Some heads will have to roll, and you will not be easy pickings as you have been prepping for this since day 1.
- Greenshift like there is no tomorrow. Your manager is going to anyways until 80% of the budget is spent and the last thing she wants is someone pooping on her parade. Always be positive, but make sure not to get caught on factual lies. Remember, you don't want to be the one easily thrown under the bus when she needs to find a scapegoat.
I think your two points are really important soft skills or work politics skills that apply transferably to any job.
Your points illustrate:
- Communicating your concerns clearly and in a timely manner,
- while also committing to do the work as it has been agreed to the best of your ability,
- and also communicating your progress clearly and regularly.
> there are often ways to fix a bad situation by breaking some explicit rules and carefully following implicit ones, and I wish I had the acuity to see these possibilities on my own.
Now extrapolate that to working in BigCo, which has tens of thousands of employees worldwide, with each country having its own unique unspoken rules and hidden undercurrents. The greatest lesson I learnt is the importance of giving people the benefit of the doubt unless they've really proven they are a bad actor, because over and over again problems turn out to be cultural at base.
All the research indicates that software developers are terrible at estimating time, but are fairly accurate at estimating relative effort. The only reliable method is to break down projects into the smallest deliverable "chunks" that you can, and then estimate the relative effort of those "chunks" compared to actual, concrete tasks that the team has completed in the past. After doing this for a while, you can use these relative effort estimates to project completion dates for new projects. This is what agile "story points" are supposed to represent, but many teams unfortunately end up with things like "2 points = 1 week", which just doesn't work.
If you use this method, it's absolutely critical that the team always focus on relative effort, and not on time. If you want it to continue to work, you also have to be very careful how you communicate with team members regarding their progress and estimated completion dates - don't tell them that their component is "late" - this will make them think in terms of time next time you're estimating something, and will ruin your estimates.
Most of the time estimate is N days, actual is N +/- 20%.
Sometimes it's N days, and you get lucky it's N/3. Instead of credit, you now have a reputation of padding estimates.
Sometimes it's N days, and you go oops, and it's 10*N. Now you are a slow incompetent.
This is why estimation is such a no-win for developers.
For areas that come closer like construction, they have centuries of prior experience and often the contractors are big enough to push back.
To use the example of this article, go to a boatyard, and say I want to 150 foot yacht with a hot tub and a 20 seat theater, and my budget is $10,000. You would get laughed out of the office.
I moved away from web development after many years of full-stack development. It almost totally destroyed the joy I had in programming.
- happy Agile team? That alone will cost you 2 days of work per week due to meetings, planning etc..
- Javascript? No, it has to be Typescript for even the most futile websites nowadays, and yes, TS adds another 20% of workload
- TDD, with a dedicated testing engineer?, no of course not, you do it yourself! Another 20% of added workload
- A shitload of tooling from linters to bundlers and whatever else that always needs some attention
- Deployement, done by a devops engineer? you must be joking right? That's also the work of the full-stack dev..
Now try to estimate how long it will take to implement a simple feature request from the PO? I always did my rough estimation, times 2. And even that was often not enough because all kinds of urgent issues needed to be resolved, so you're kind of lacking all the time which is a great recipe for burnout. I'm done with it.
I'm not sure how to avoid answering these loaded questions though. In my experience trying to 'negotiate' with people like that: they will just ask you the same question in 5 different ways and it will eventually get so uncomfortable you'll tell them anything to move on.
Another option might be to never do business with cash-starved companies. Founders and execs will be constantly worried about running out of money, acquiring customers, and pulling off miracles, and all that stress will trickle down on you probably for no real benefits.
A key question to ask an employer then is how much 'run way' / money they have left? Or whether they have a revenue source. It seems fairly probing, but realistically, not everyone wants to invest heavily in a company that might not even be around in 6 months.
> I'm not sure how to avoid answering these loaded questions though. In my experience trying to 'negotiate' with people like that: they will just ask you the same question in 5 different ways and it will eventually get so uncomfortable you'll tell them anything to move on.
Once you realize what's going on, you can turn that into a game.
Do you know the game where somebody asks you a bunch of questions, and they you allowed to answer with anything except "yes" or "no"? It's very hard to do, because it's such an ingrained habit.
If you really want to not give an estimate, make it a game to not give one.
But, give them something else instead. Work out a bunch of questions about uses cases / data volume / whatever, and say "if those were answered, we could build a prototype in a few days that would let us make a more reliable estimate" or something like that.
Another comment: coming up with good estimates is work. The other day somebody asked me if I could come up with a rough estimate for a (poorly specified, IMHO) project, and my answer was: no, I don't have time. If you need it anyway, formulate it as a task in Jira, so that it gets prioritized along with all my other work.
(Fun fact: we estimate our tasks in story points, so then we estimated how much time it would take us to come up with an estimate... :D )
On the flipside, necessity is the mother of invention. How many behemoth companies do we see now that sit on their hands and don't bother innovating at all? There's surely a balance to strike here between desperate and panicking and fat and lazy.
This sort of thing is why engineers are paid way too much, and way too little, and why so many see the profession as a young man's game. You're just not the master of your own fate. Sure, the computers are predictable (for some value of "predictable"...) but management is not---even if they try to be.
I've been mentally moving away from being paid to write software. I can see writing it for my own use, even professional use---I just don't want to be on the hook for ever-changing requirements, decided by people who are often kind, but not, in the end, competent. The best managers understand this and will give you leeway, but this is not a sustainable, repeatable thing. It lives and dies on one relationship.
I wonder sometimes: what if we just all stopped writing software one day, and started just using it? Writing software is a bad deal in a lot of ways---hard, socially isolating, etc---while using it is amazing---the computer does the work for you!
Don't get me wrong, I spent an hour at work today presenting on Lisp macros and loved every minute of it. But a dev career, for many, means a capped income and a razor's edge of apparent competence.
What jobs don't have drawbacks though? I have been thinking a bit about this:
1. Doctor: In many cases, throw away your life and work 60+ hours and also you need to specialize and study long and hard.
2. Laywer: I don't know enough about the profession. The job doesn't transfer well to other countries.
3. Consultant: 60+ hour days are the norm, interviews tend to be based on quick thinking in the high school arena. To pass the interview, you need to have excellent and super quick high school level knowledge of: math, logic, social and political skills. This sound denigrating but I found it tough to do this quick.
4. Investment banker: 100 hours
5. Construction worker: your body will thank you later (/sarcasm).
Programmers have a certain set of advantages and disadvantages (I agree with your disadvantages), but how is it worse than other white collar jobs?
You might consider whether you are ready to go into management. We know the trope of the engineer forced into management, who did not want that job. But I think the opposite is just as common -- great engineers, who would also be great managers, that never (want to) make the transition. Understandable -- a good management day seems less enjoyable than a good code writing day. But I bet most companies, careers, and products would be the better for it.
My understanding is that it is harder for engineers to estimate how long a job takes, than to do the job. That is to say, that the complexity of doing a time estimation task is higher than the complexity of task you are estimating. I read this in the book "Making Software: What really works and Why we believe it" [0]. The assumption in the article is that an engineer can produce reliable estimates for complex work, and appropriately guide their managers, and don't think that's true.
Estimates are hard because you are literally making something that has never been made before. Yes, you have experience with sub parts and similar patterns but not the situation of this timeline, this team, this technical debt, etc.
Too often the request is along the lines of "How long does it take to build a house?" not "How long will it take to build the house in these blueprints with these systems on top of a mountain that is also impervious to mudslides?". People can generally estimate the former but the latter is all about the unknown details.
I don’t think the article assumes an engineer can produce reliable estimates for complex work; in fact it claims the opposite:
> But the reality is that if you can make a probabalistically accurate estimate, then its likely that the task should have been automated by some other means already. In other words, its easy to estimate a task that essentially amounts to copy and pasting some well known CRUD API end point patterns, but any even remotely creative or novel work is almost guaranteed to be totally unknown.
I recently estimated a task would take 30 minutes.
It took 40 minutes to do that estimate.
But, TBF, several important decisions were made during the process of estimating, such as what should be excluded from the task, basic organization, and some research.
BTW the task being estimated was doing time estimates for a project (which came out to be 2-3 weeks).
In that case it seems like it would be better for PM's to take the role of task estimation. Of course this only works if they are also the ones accountable for inaccurate estimates.
A few years ago I was brought in lead a team in a division of a public company. Our mandate was to replace a major component of an enterprise system that you couldn't quite call legacy because they'd never actually gotten around to replacing the real legacy system with it.
My initial proposal was to build it incrementally: tackle the obviously critical functionality to get something working and ship it, then define and implement additional features on an ongoing basis. I had hoped this would fly, since the company talked a big game about being agile.
But no dice. We had to have a complete plan for all the features currently supported (some of them of dubious value, many of them incomplete and inconsistent in specification), and it had to come with a specific timetable. Under protest, the team and I worked hard to produce high-level estimates, and ended up basically guessing that the thing would take six months.
No dice, it had to be delivered in three. I made a suggestion along the lines of the strategy suggested in the article -- we could identify a subset of features that the team would be comfortable could be delivered within the deadline. It would probably be stronger, I argued, since it wasn't clear that all the added weight added much value.
No dice: everything was priority one. I pointed out something like, "you can't fight the laws of physics". I apparently gave the impression that we'd get it done anyway, though my memory of the conversation was that I was sternly disagreeable.
Our team goes ahead and starts implementing items from a value-prioritized backlog. Fast forward three months, and we have a working system that supports the most important use-cases. We considered it past ready to ship, knowing we'd need to keep iterating. The response from management, predictably, was frustration at the missing features, despite the advance warnings.
Management goes into "high-pressure" mode, and for the next three months I do my best to keep the team insulated. After more or less six months of total development time, we finally replace the prior component. All the users agree that it has far fewer bugs. I and many of my team members grumble that it has far too many, on account of the fact that we weren't given the chance to ship the minimum viable product when it was ready.
I'm not really sure what the moral of the story is.
‘we just need a rough estimate, we won’t hold you to it’.
‘Ok based on the 2 minute conversation we just had I think about 3 months’
‘What! That seems way too long’
Other times I’ve been asked for estimates on features even though there is a hard deadline due to some external factor. I really fail to see the point of estimating anything when there is already a decided upon end date. Anyway I usually point out that they will need to just put the features in order of importance and I’ll work down the list. And they should start thinking about what can be cut. This usually leads to protests of ‘we have already cut everything we can’. But I have to laugh as we get closer to the deadline that suddenly not every feature is as important as was originally thought and magically get cut.
If managers come to developers with problems that are NP complete, no matter how well they transfer the idea/vision, or set time expectations, the problem cannot be solved efficiently. And, unless the developers have been trained in CS/Math they may not even understand that what they are being asked to do cannot be done.
For example, say a manager has an idea to find the largest group of friends in a massive social network. He wants a developer to write an app for that and has a 20K budget and 2 months. You could not write this app with 10 times that budget or time.
How can you determine which group of friends is the largest?
There's no substitute for technically competent management, period. If you don't have it, you will be limited, and its something you just have to accept -- as in, make that a high priority question to ask about when interviewing for your next job. Further, its not that software developers are bad at estimating. Its that software estimation is only useful when taken as a very rough guide for strategic planning. You need to know if something will take 2 weeks or 2 years. You don't need to know if it will take 2 weeks or 4 weeks. If you think you do (as most shops I've worked do) -- you are micro managing your team. The sooner we start calling it that, the sooner we can all accepts its as much an anti-pattern in software development as it is every other facet of business.
In short, estimate roughly. Agree to high level deliverables. Assert that technical management needs some control over strategic business decisions, and flexibility in implementation and timelines. If you can't get those things, temper your expectations, or look for a job with more competent leadership. The last point has become my guiding principle. Stop thinking you can "fix" an org from the bottom. You can only fix an org if they hire / promote you to do so, and empower you as such. That's what you ask for (or look for in your managers).
Project plans and estimates are usually fantasies made only to give executives a sense of control. For people who are actually trying to get something done in the world, I think the better thing to do is give them actual control via short-cycle processes and frequent delivery.
At my last startup, we got very good at releasing early and often. What would have been projects elsewhere got broken down into very small releasable units; our average story size was under a day. Very occasionally my cofounder would have us ballpark estimate two different batches of work using arbitrary points, just so he could get a sense of the relative cost of two paths. But we never estimated in terms of dates.
This worked for us because he really took advantage of the speed of iteration. Almost everything we built came with a question. In the next user test, would users understand it? Would they want to use it? Did they react as expected? When we sent some traffic to it, did people engage? Did the right people engage? Did they return to use it later, indicating value? Etc, etc.
The answers to those questions would drive what we did next. Because our goal wasn't to build features, but to make things happen in the real world. And once you get used to continuous learning, it's obvious that planning too far in advance is wasteful. All of the brilliant ideas you had a month ago weren't based on what you learned in the last month. Eventually, sensible people learn to stop producing a lot of plans that never get fully used. And, of course, to stop asking for estimates on them.
One of the most painful types of people to work with is the CEO / founder who got lucky with a product early and came to believe they're a "product guy".
As in: My early product hit whatever trend / wave / need therefore I must be a product guy (and not just lucky).
The product guy has a preternatural ability to understand what the masses want. Watching them work -- witnessing their process -- is something to behold. They will steer products in a direction regardless of cost, complexity of likely outcome.
The outcome, quite often, is to tank their company. Since they don't understand why they were successful in the first place, it's very likely that their success won't last.
But if you're along for the ride, wow, expect the following:
1. You're the greatest (available) engineer we've ever encountered, building super-complicated XYZ is going to take this company to the next level!
2. This is taking much longer than expected and isn't matching up with our expectations but I'm 100% sure of my vision because I'm a product guy.
3. We're running out of money (because the market conditions that gave us early success have changed) and super-complicated XYZ isn't going to rescue us -- because you're a worthless piece of shit of an engineer!
See what happened there?
They're sometimes hard to distinguish from a vanilla bullshit artist. The bullshit artist will tell you how well capitalized he is, tell you he only wants the best (meaning he thinks you're expensive) and then try to slowly whittle your sense of self worth down until you get "the offer":
The offer is game-changing, life-altering for you: Instead of continuing to pay you with money, they're going to start paying you with magic pixie dust. The magic pixie dust will make you rich "when everything comes together."
When you tell bullshit artist that you don't work for magic pixie dust, that's when you learn that, in fact, you're a worthless piece of shit of an engineer.
I actually respect the bullshit artist more: They're bullshitting other people but they know they're full of shit. Product guys, depressingly, bullshit themselves.
> It is our belief that over the 30 plus years of commercial computing has developed a series of sophisticated political games that have become a replacement for estimation as a formal process.
Unrelated: can we not do this thing with the whole left side of the screen being one static image? It's really distracting.
Timelines kill me. When I'm half way through a project I can tell you exactly when it's going to be done. Or I can plan the shit out of it upfront and give you an accurate timeline but that doesn't fly because you need the timeline upfront. I like working - can't I just work till it's properly done?
My usual way of doing this, is by spitballing an estimate with the team, multiplying it by 2.5 or 3, inventing some meaningless milestones and gantt-charts and presenting it to an outraged management. Then we haggle down to something realistic + a thin buffer. The management has a sense of control, we have realistic deadlines.
Caveat: I work in the financial industry, where the complexity of writing a compiler and whipping up a Tableau dashboard are perceived to be equal.
I’ve been participating in build pipeline work for quite a long time now, and fairly early on I noticed a pattern. If the build takes ten minutes on your local box then the tempo of code-build will tend to be 20 minutes, even when you are just fixing typos.
This is partly why some people are obsessed with very short builds. Some said seven minutes was ideal to avoid the developer getting preempted. Then it was three. Now some shoot for one.
I would bring this up and others would say they had noticed the same thing, but none of us knew why this phenomenon happens.
Then it hit me: it’s just Hofstadler’s Law playing out in the small. If you think you have five minutes, you will start something that you estimate will be five minutes, but it will take you ten, either because you were wrong or you get distracted.
There aren’t a lot of tasks that take one minute, and even if you’re wrong you only lose one minute.
When I was a young programmer in the early nineties, an old IBM programmer told me 'Don't accept the invitation to fail.' Good advice but what if the goal posts are moved. Best then to keep positive and communicate early and often about how the changes will affect the schedule. Work together as a team and get your manager to help you and that'll go a long way rather than keeping 'obvious' things to yourself and catching management off guard when work slips.
> Inside this system lurks the biggest difference between the IT and Hollywood: movie industry exists for more than a hundred years already, they had enough time to develop and establish best practices and to prove them in practice to such an extent as to tell everyone: this is how you need to do movies; and everyone can trust it works, because it has worked for the whole industry for decades already. IT industry is very young, and exists for a few decades.
(Although, IT is technically as old as the cuneiform-inscribed clay tablet, eh?)
Specifically to this discussion, they (Hollywood) can set expectations reliably because have enough shared baseline experience of how long things take.
One way to interpret this is to ask yourself, of some new change in process or technology, "Will this stabilize delivery times?" But to even begin to answer that kind of question you have to establish reasonable ways to map between work done and results delivered, and then set up and track metrics. (Which is easier said than done.)
- - - -
One problem unique to IT is the "interpretability" of our end products. Anyone can watch a movie, but most software requires at least some training to understand and use.
[+] [-] lostdog|6 years ago|reply
But I noticed that some people handled the situation fine. They stayed on management's good side, even though they were failing to deliver along with the rest of us. I will try to distill what I observed them do:
1) They did not fight on the estimates. If a manager forced them into a certain timeline, they registered their disagreement and just accepted the new timeline. I think they realized that fighting would just make the manager judge them less capable. (Pick your battles, eh?).
2) When the schedule slipped, they would communicate it in a way that made them seem more competent instead of less. The explanation usually had three parts: unforeseen events kept us from hitting the deadline, we accomplished some great things in the meantime, and here's why we're in great shape to hit the next deadline! For example: "When we made this schedule, we did not realize that AWS nodes were so unreliable! Despite this, our team has made incredible progress on implementing a fast method of storage and solid compression! We have reworked the schedule to reflect this new information, and we are already on track for the first milestone!"
It's possible that this trick just worked for the specific situation at $OldJob, but I really enjoyed learning it. They seemed to understand that certain explicit rules were not important, but that other unspoken rules needed to be followed. Are accurate estimates important? It depends on the situation! Sometimes wrong estimates can be more valuable than correct ones. Construction companies give wrong estimates all the time in order to win projects. Is staying on good terms with your boss important? Yes! Even if they are total shitbags, being adversarial won't help, only leaving will. These people demonstrated that there are often ways to fix a bad situation by breaking some explicit rules and carefully following implicit ones, and I wish I had the acuity to see these possibilities on my own.
[+] [-] PragmaticPulp|6 years ago|reply
> 2) When the schedule slipped, they would communicate it in a way that made them seem more competent instead of less.
In other words: Enable the bad estimates, then externalize accountability when they can't be achieved. In the past, I called these people "blameless go-getters" because they were always first to volunteer to take on work, but somehow it became everyone else's fault when they failed. If management is asleep at the wheel, it's a win-win situation for them.
You're exactly right that this method works in many companies. Like you said, it's all about understanding what the company actually values. When unrealistic schedules are forced on teams, the exact date might not be important. Instead, it might be leadership's dysfunctional way of emphasizing focus, urgency, and quick iteration. The savvy engineers and managers know how to put on a show that hits these key points while steadily delivering progress in the background.
Still, there's no escaping the fact that this is dysfunctional. More importantly, it doesn't have to be like this. It's eye-opening to move from a dysfunctional company like you described toward a company that values honest communication and understands engineering project management at the executive leadership level. When hiring someone out of a dysfunctional company like you described, it can take some time to break them of bad habits around schedule misdirection and estimation dishonesty.
[+] [-] wpietri|6 years ago|reply
To see it in contrast, think for a moment about a well-run hospital. At a hospital, the people who make the key decisions about cases are medical professionals, not managers. If a surgery was supposed to take 4 hours but takes 12, well, that's how long the surgery took. Everybody recognizes that the 4-hour number was an estimate, and estimates are not commitments. The most important thing is patient health, not manager feelings. Managers help organize the work, but they do not control the work.
I would love to see software development become a true profession, where stroking manager egos by making them always feel correct and in control is not the most important thing.
[+] [-] mannykannot|6 years ago|reply
[+] [-] mcv|6 years ago|reply
[+] [-] PeterStuer|6 years ago|reply
- CYA above all. The chances of a significant project succeeding in such an environment are pretty slim. Prepare your umbrella from the get go for when the inevitably will hit the fan. Some heads will have to roll, and you will not be easy pickings as you have been prepping for this since day 1.
- Greenshift like there is no tomorrow. Your manager is going to anyways until 80% of the budget is spent and the last thing she wants is someone pooping on her parade. Always be positive, but make sure not to get caught on factual lies. Remember, you don't want to be the one easily thrown under the bus when she needs to find a scapegoat.
I chose not to stay in such environments.
[+] [-] learnstats2|6 years ago|reply
Your points illustrate: - Communicating your concerns clearly and in a timely manner, - while also committing to do the work as it has been agreed to the best of your ability, - and also communicating your progress clearly and regularly.
This ought to be valued by any employer.
[+] [-] saalweachter|6 years ago|reply
Task X will take 2 weeks.
Task X will take 2 weeks of development time and 6 weeks to test, validate, productionize and roll out.
Task X would take 2 weeks if we drop everything else we're doing, but because we can't, it will take about 12-18 weeks to finish.
[+] [-] JTbane|6 years ago|reply
[+] [-] growlist|6 years ago|reply
Now extrapolate that to working in BigCo, which has tens of thousands of employees worldwide, with each country having its own unique unspoken rules and hidden undercurrents. The greatest lesson I learnt is the importance of giving people the benefit of the doubt unless they've really proven they are a bad actor, because over and over again problems turn out to be cultural at base.
[+] [-] alexhutcheson|6 years ago|reply
If you use this method, it's absolutely critical that the team always focus on relative effort, and not on time. If you want it to continue to work, you also have to be very careful how you communicate with team members regarding their progress and estimated completion dates - don't tell them that their component is "late" - this will make them think in terms of time next time you're estimating something, and will ruin your estimates.
[+] [-] JJMcJ|6 years ago|reply
Manager says I want an A that does B? How long?
Most of the time estimate is N days, actual is N +/- 20%.
Sometimes it's N days, and you get lucky it's N/3. Instead of credit, you now have a reputation of padding estimates.
Sometimes it's N days, and you go oops, and it's 10*N. Now you are a slow incompetent.
This is why estimation is such a no-win for developers.
For areas that come closer like construction, they have centuries of prior experience and often the contractors are big enough to push back.
To use the example of this article, go to a boatyard, and say I want to 150 foot yacht with a hot tub and a 20 seat theater, and my budget is $10,000. You would get laughed out of the office.
[+] [-] blobs|6 years ago|reply
[+] [-] bigYahnz|6 years ago|reply
[deleted]
[+] [-] Uptrenda|6 years ago|reply
Another option might be to never do business with cash-starved companies. Founders and execs will be constantly worried about running out of money, acquiring customers, and pulling off miracles, and all that stress will trickle down on you probably for no real benefits.
A key question to ask an employer then is how much 'run way' / money they have left? Or whether they have a revenue source. It seems fairly probing, but realistically, not everyone wants to invest heavily in a company that might not even be around in 6 months.
[+] [-] perlgeek|6 years ago|reply
Once you realize what's going on, you can turn that into a game.
Do you know the game where somebody asks you a bunch of questions, and they you allowed to answer with anything except "yes" or "no"? It's very hard to do, because it's such an ingrained habit.
If you really want to not give an estimate, make it a game to not give one.
But, give them something else instead. Work out a bunch of questions about uses cases / data volume / whatever, and say "if those were answered, we could build a prototype in a few days that would let us make a more reliable estimate" or something like that.
Another comment: coming up with good estimates is work. The other day somebody asked me if I could come up with a rough estimate for a (poorly specified, IMHO) project, and my answer was: no, I don't have time. If you need it anyway, formulate it as a task in Jira, so that it gets prioritized along with all my other work.
(Fun fact: we estimate our tasks in story points, so then we estimated how much time it would take us to come up with an estimate... :D )
[+] [-] jimbob45|6 years ago|reply
[+] [-] dropit_sphere|6 years ago|reply
I've been mentally moving away from being paid to write software. I can see writing it for my own use, even professional use---I just don't want to be on the hook for ever-changing requirements, decided by people who are often kind, but not, in the end, competent. The best managers understand this and will give you leeway, but this is not a sustainable, repeatable thing. It lives and dies on one relationship.
I wonder sometimes: what if we just all stopped writing software one day, and started just using it? Writing software is a bad deal in a lot of ways---hard, socially isolating, etc---while using it is amazing---the computer does the work for you!
Don't get me wrong, I spent an hour at work today presenting on Lisp macros and loved every minute of it. But a dev career, for many, means a capped income and a razor's edge of apparent competence.
[+] [-] mettamage|6 years ago|reply
1. Doctor: In many cases, throw away your life and work 60+ hours and also you need to specialize and study long and hard.
2. Laywer: I don't know enough about the profession. The job doesn't transfer well to other countries.
3. Consultant: 60+ hour days are the norm, interviews tend to be based on quick thinking in the high school arena. To pass the interview, you need to have excellent and super quick high school level knowledge of: math, logic, social and political skills. This sound denigrating but I found it tough to do this quick.
4. Investment banker: 100 hours
5. Construction worker: your body will thank you later (/sarcasm).
Programmers have a certain set of advantages and disadvantages (I agree with your disadvantages), but how is it worse than other white collar jobs?
[+] [-] cloverich|6 years ago|reply
[+] [-] arichard123|6 years ago|reply
[0] https://www.amazon.co.uk/Making-Software-Really-Works-Believ...
[+] [-] snarf21|6 years ago|reply
Too often the request is along the lines of "How long does it take to build a house?" not "How long will it take to build the house in these blueprints with these systems on top of a mountain that is also impervious to mudslides?". People can generally estimate the former but the latter is all about the unknown details.
[+] [-] jtc331|6 years ago|reply
> But the reality is that if you can make a probabalistically accurate estimate, then its likely that the task should have been automated by some other means already. In other words, its easy to estimate a task that essentially amounts to copy and pasting some well known CRUD API end point patterns, but any even remotely creative or novel work is almost guaranteed to be totally unknown.
[+] [-] hyperpallium|6 years ago|reply
It took 40 minutes to do that estimate.
But, TBF, several important decisions were made during the process of estimating, such as what should be excluded from the task, basic organization, and some research.
BTW the task being estimated was doing time estimates for a project (which came out to be 2-3 weeks).
[+] [-] skohan|6 years ago|reply
[+] [-] couchand|6 years ago|reply
My initial proposal was to build it incrementally: tackle the obviously critical functionality to get something working and ship it, then define and implement additional features on an ongoing basis. I had hoped this would fly, since the company talked a big game about being agile.
But no dice. We had to have a complete plan for all the features currently supported (some of them of dubious value, many of them incomplete and inconsistent in specification), and it had to come with a specific timetable. Under protest, the team and I worked hard to produce high-level estimates, and ended up basically guessing that the thing would take six months.
No dice, it had to be delivered in three. I made a suggestion along the lines of the strategy suggested in the article -- we could identify a subset of features that the team would be comfortable could be delivered within the deadline. It would probably be stronger, I argued, since it wasn't clear that all the added weight added much value.
No dice: everything was priority one. I pointed out something like, "you can't fight the laws of physics". I apparently gave the impression that we'd get it done anyway, though my memory of the conversation was that I was sternly disagreeable.
Our team goes ahead and starts implementing items from a value-prioritized backlog. Fast forward three months, and we have a working system that supports the most important use-cases. We considered it past ready to ship, knowing we'd need to keep iterating. The response from management, predictably, was frustration at the missing features, despite the advance warnings.
Management goes into "high-pressure" mode, and for the next three months I do my best to keep the team insulated. After more or less six months of total development time, we finally replace the prior component. All the users agree that it has far fewer bugs. I and many of my team members grumble that it has far too many, on account of the fact that we weren't given the chance to ship the minimum viable product when it was ready.
I'm not really sure what the moral of the story is.
[+] [-] anthonyoconnor|6 years ago|reply
‘we just need a rough estimate, we won’t hold you to it’.
‘Ok based on the 2 minute conversation we just had I think about 3 months’
‘What! That seems way too long’
Other times I’ve been asked for estimates on features even though there is a hard deadline due to some external factor. I really fail to see the point of estimating anything when there is already a decided upon end date. Anyway I usually point out that they will need to just put the features in order of importance and I’ll work down the list. And they should start thinking about what can be cut. This usually leads to protests of ‘we have already cut everything we can’. But I have to laugh as we get closer to the deadline that suddenly not every feature is as important as was originally thought and magically get cut.
[+] [-] w8rbt|6 years ago|reply
For example, say a manager has an idea to find the largest group of friends in a massive social network. He wants a developer to write an app for that and has a 20K budget and 2 months. You could not write this app with 10 times that budget or time.
How can you determine which group of friends is the largest?
https://en.wikipedia.org/wiki/Clique_problem
https://en.wikipedia.org/wiki/List_of_NP-complete_problems
[+] [-] cloverich|6 years ago|reply
In short, estimate roughly. Agree to high level deliverables. Assert that technical management needs some control over strategic business decisions, and flexibility in implementation and timelines. If you can't get those things, temper your expectations, or look for a job with more competent leadership. The last point has become my guiding principle. Stop thinking you can "fix" an org from the bottom. You can only fix an org if they hire / promote you to do so, and empower you as such. That's what you ask for (or look for in your managers).
[+] [-] wpietri|6 years ago|reply
At my last startup, we got very good at releasing early and often. What would have been projects elsewhere got broken down into very small releasable units; our average story size was under a day. Very occasionally my cofounder would have us ballpark estimate two different batches of work using arbitrary points, just so he could get a sense of the relative cost of two paths. But we never estimated in terms of dates.
This worked for us because he really took advantage of the speed of iteration. Almost everything we built came with a question. In the next user test, would users understand it? Would they want to use it? Did they react as expected? When we sent some traffic to it, did people engage? Did the right people engage? Did they return to use it later, indicating value? Etc, etc.
The answers to those questions would drive what we did next. Because our goal wasn't to build features, but to make things happen in the real world. And once you get used to continuous learning, it's obvious that planning too far in advance is wasteful. All of the brilliant ideas you had a month ago weren't based on what you learned in the last month. Eventually, sensible people learn to stop producing a lot of plans that never get fully used. And, of course, to stop asking for estimates on them.
[+] [-] yuoiyuio_UYO|6 years ago|reply
As in: My early product hit whatever trend / wave / need therefore I must be a product guy (and not just lucky).
The product guy has a preternatural ability to understand what the masses want. Watching them work -- witnessing their process -- is something to behold. They will steer products in a direction regardless of cost, complexity of likely outcome.
The outcome, quite often, is to tank their company. Since they don't understand why they were successful in the first place, it's very likely that their success won't last.
But if you're along for the ride, wow, expect the following:
1. You're the greatest (available) engineer we've ever encountered, building super-complicated XYZ is going to take this company to the next level!
2. This is taking much longer than expected and isn't matching up with our expectations but I'm 100% sure of my vision because I'm a product guy.
3. We're running out of money (because the market conditions that gave us early success have changed) and super-complicated XYZ isn't going to rescue us -- because you're a worthless piece of shit of an engineer!
See what happened there?
They're sometimes hard to distinguish from a vanilla bullshit artist. The bullshit artist will tell you how well capitalized he is, tell you he only wants the best (meaning he thinks you're expensive) and then try to slowly whittle your sense of self worth down until you get "the offer":
The offer is game-changing, life-altering for you: Instead of continuing to pay you with money, they're going to start paying you with magic pixie dust. The magic pixie dust will make you rich "when everything comes together."
When you tell bullshit artist that you don't work for magic pixie dust, that's when you learn that, in fact, you're a worthless piece of shit of an engineer.
I actually respect the bullshit artist more: They're bullshitting other people but they know they're full of shit. Product guys, depressingly, bullshit themselves.
[+] [-] brianpgordon|6 years ago|reply
> It is our belief that over the 30 plus years of commercial computing has developed a series of sophisticated political games that have become a replacement for estimation as a formal process.
Unrelated: can we not do this thing with the whole left side of the screen being one static image? It's really distracting.
[+] [-] niknikson|6 years ago|reply
[+] [-] jevgeni|6 years ago|reply
Caveat: I work in the financial industry, where the complexity of writing a compiler and whipping up a Tableau dashboard are perceived to be equal.
[+] [-] hinkley|6 years ago|reply
This is partly why some people are obsessed with very short builds. Some said seven minutes was ideal to avoid the developer getting preempted. Then it was three. Now some shoot for one.
I would bring this up and others would say they had noticed the same thing, but none of us knew why this phenomenon happens.
Then it hit me: it’s just Hofstadler’s Law playing out in the small. If you think you have five minutes, you will start something that you estimate will be five minutes, but it will take you ten, either because you were wrong or you get distracted.
There aren’t a lot of tasks that take one minute, and even if you’re wrong you only lose one minute.
[+] [-] nstart|6 years ago|reply
If there's a single line you want to take away from this wonderful essay, it would be this.
Parkinson's law usually takes care of the rest once the resources have been agreed upon.
[+] [-] djmips|6 years ago|reply
[+] [-] carapace|6 years ago|reply
I'm grateful for "reader mode".
- - - -
There's a very interesting book, " Hollywood Secrets of Project Management Success" that details their system from an IT point of view. Here's a decent review: https://community.dynamics.com/nav/b/navigateintosuccess/pos...
> Inside this system lurks the biggest difference between the IT and Hollywood: movie industry exists for more than a hundred years already, they had enough time to develop and establish best practices and to prove them in practice to such an extent as to tell everyone: this is how you need to do movies; and everyone can trust it works, because it has worked for the whole industry for decades already. IT industry is very young, and exists for a few decades.
(Although, IT is technically as old as the cuneiform-inscribed clay tablet, eh?)
Specifically to this discussion, they (Hollywood) can set expectations reliably because have enough shared baseline experience of how long things take.
One way to interpret this is to ask yourself, of some new change in process or technology, "Will this stabilize delivery times?" But to even begin to answer that kind of question you have to establish reasonable ways to map between work done and results delivered, and then set up and track metrics. (Which is easier said than done.)
- - - -
One problem unique to IT is the "interpretability" of our end products. Anyone can watch a movie, but most software requires at least some training to understand and use.