top | item 4510943

Why I'm done with Scrum

194 points| hkarthik | 13 years ago |lostechies.com | reply

157 comments

order
[+] fingerprinter|13 years ago|reply
When I took over development of a extremely dysfunctional engeering team at an established startup with 100 people, the first thing I did (after watching for a bit to learn) was put in scrum. It worked. And it worked really, really well.

2 years later, it didn't work anymore. The team had matured, the organization itself had adapted to the leaner processes and mindset and, in time, the actual scrum process was too much. It was time to change again...

And that is something I think most agile advocates don't realize. Agile should be viewed as an organizational tool that has prescripted and prescribed rules that work in general, but maybe don't work in the specific. It is great if you need something forceful with books or documentation in the beginning, but in time, the organization should mature to the point where agile/scrum/whatever is actually too much.

One last thought. I highly recommend people look at agile for the times when you need a drop in process....but there is absolutely no subsitute for hiring quality people capable of making good decisions. Agile will get more out of bad developers, IMO, but having nothing and good developers you trust is always going to be more productive.

[+] debacle|13 years ago|reply
I think your last point is key. You can't use process to turn a bad programmer into a mediocre one - any process that does will also turn a great programmer into just a mediocre one.
[+] shanemhansen|13 years ago|reply
Respectfully, I think that an established startup with 100 people is a bit of an oxymoron.

I know that no company ever wants to think of itself as "big and established", in fact my previous company had been around for 13 years, had hundreds of millions in revenue and almost 1k employees and they still liked to pretend they were a disruptive startup.

[+] sandGorgon|13 years ago|reply
That was an excellent observation. I like to explain the phenomenon you observed using queuing theory - As time went on, each of you (and you yourself) were able to estimate tasks very, very accurately. I'd like to think that this was a period of low attrition.. but even if it was not, you were able to model the behavior of a 100 employees (a large enough set) to come up with good estimates.

An agile/scrum process is designed to work with tasks of inaccurate estimates - the whole business of story points is designed around that fact. Since your underlying phenomenon changed, the process was no longer optimal.

I'm not sure if you experimented with your own "lean" inventions beyond agile - single queues, multiple queues, queues with an artificial stop signal to reduce variance, etc. - but it would have been interesting.

[+] shaunxcode|13 years ago|reply
Agile should also be subject to Agile. I like to think of it in terms of the Viable System Model in which the system has the potential to change itself based upon feedback. Put another way - make sure your implementation of Agile/Scrum supports tail call optimization and macro expansion or face the reality of being stuck in BlubScrum.
[+] m_st|13 years ago|reply
I fully agree with your post. But do mind sharing what process you changed to then? Did it and does it still work?
[+] boomzilla|13 years ago|reply
Agreed. When you don't have collective excellence anymore, you are left with process.
[+] martincmartin|13 years ago|reply
I've done a bunch of reading about Scrum. If you read between the lines, you realize that Scrum was created and popularized by consultants who go into dysfunctional teams/organizations, and tries to fix the worse problems. For example, the idea of a sprint is for a team to be able to work for at least a couple weeks on a single thing, without people being asked to work on other "small" projects, or without the entire direction of development changing every week.

Once the team is functioning well, you can usefully relax a lot of the aspects of Scrum.

[+] mizhi|13 years ago|reply
After a bit of experience with scrum, one of the things I"ve come to appreciate is that it's not a one-size-fits-all methodology. I think the spirit of scrum (break work into chunks, have working code at each stake in the ground, define a project in terms of user stories, etc) is more important than the letter of the law, and being able to tailor the process to your current reality is more valuable than dogmatic adherence to some prescription.
[+] gaius|13 years ago|reply
Scrum is by and for management, not developers. Want your devs in at 9am? No problem, hold the standup then. Want to easily replace your dev team with an outsourced one? No problem, a Scrum team is a black box as far as the rest of the organization is concerned, one with a well-documented interface. Etc, etc.

Ir's insidious because it tricks devs into thinking it's for them.

[+] ams6110|13 years ago|reply
I think you're being a bit generous. The consultants who sell "Agile" to dysfunctional organizations are the same ones who sold every fad methodology that preceded it to those same dysfunctional organizations.
[+] sjtgraham|13 years ago|reply
This is a really interesting insight. I have seen so many instances of prescriptive Scrum with no explanation as to why that it's appropriate, i.e. Cargo Culting.
[+] WickyNilliams|13 years ago|reply

    I've done a bunch of reading about Scrum
Without trying to be inflammatory, this is no way qualifies you to speak with any authority on Scrum, or indeed any topic where your experience amounts to nothing more than a bunch of reading.

That is not to say that you don't make any valid points. I think the fundamental tenets could be relaxed over time, but I guess with some organisations when you start removing some of the restrictions you'll eventually remove more and more until you're back where you started.

[+] debacle|13 years ago|reply
> With Scrum, there is an explicit commitment ... on what stories are going to be delivered within the sprint,

No, there isn't. You adhere to your burn-down, not to your feature set. Scrum is time driven, not task driven. The whole idea is to become better at estimation so that Scrum appears task driven, when really it's just because your team is that good at estimating.

> Iteration planning meetings are seriously expensive.

Then you're doing it wrong (god, I feel awful for saying that). If you're spending that much time in pre-sprint planning, people aren't bringing enough to the table and you're not defining your problems well enough before trying to solve them.

> I hate the term “scrum-but”.

Ironic that that sounds like exactly what the environment you were working in sounds like.

> This might be a bit controversial, but the big difference between Scrum and things like Lean Software Development for me were the difference of focusing on process versus delivery.

I think this is a "forest for the trees" issue. Scrum is about abstracting the process into a rigid and lightweight framework so you only have to think about it, at most, fifteen minutes a day (unless you have the misfortune of being the Scrum master/cat wrangler)

> It can’t, and it’s not a problem with your organization if it doesn’t. It’s a problem with Scrum.

That's a non-sequitur. It's a problem with neither. If my company can't implement a waterfall methodology because our clients require faster iteration, that doesn't mean there's a problem with either - it's just not a good fit.

But if your company tries to implement Scrum and throws it out six weeks or six months later because it's "too hard" or "bad," my assumption is that real Scrum probably wasn't happening.

Will Scrum work for everyone? Fuck no. Scrum is hard. It requires the kind of buy in that, if you've got it, you probably don't need Scrum anyway.

Can Scrum work for everyone? Probably. Scrum is only hard when you've got conflicting goals. If you have a smart enough team, you can tweak your sprint lengths from as little as a few days to as long as six to eight weeks, and after you've been on Scrum for a few sprints your planning meetings should take an hour or two, tops.

[+] sethg|13 years ago|reply
In my experience (six months at a job that does Scrum and, I think, does it very well), the thing that slows down iteration planning meetings is when the product manager hands down a one-sentence feature description, the engineers say “we can’t size that, it’s too vague”, and then you need a five-to-fifteen-minute discussion in order to expand that one sentence into something resembling a spec.
[+] tosh|13 years ago|reply
“A bad system, will defeat a good person, every time.” -- Deming

I can no longer count how often I heard "you're doing it wrong" when people explain why scrum didn't work for them or when they feel there needs to be something that works 'better'.

Getting Scrum right is hard. Managing huge backlogs is hard. Doing time-based sprint plannings is hard, especially if you are working on stuff that is either new or new to your team.

If you are doing routine work you might as well use waterfall or "jfdi" as your approach. Burn-down charts, velocity and sprint planning doesn't help if you just don't know where the next problem will be or when your team variation changes constantly (people move to other teams, new people join, people get sick, …).

I've found that changing/being able to change the 'system' (scrum) to something else that works better (agile/lean to its roots, kanban, different stand-up meeting format, evidence based time-tracking, value-flow …) is way more effective than changing the people to become 'better' at Scrum and doing it less 'wrong'.

YMMV

[+] wccrawford|13 years ago|reply
>It requires the kind of buy in that, if you've got it, you probably don't need Scrum anyway.

I agreed up until this. I worked for a company that was doing waterfall and miserable. Things were getting done, but it was a sloppy mess. Clearly we didn't have our stuff together.

Our manager went and took scrum classes, and then we spent a few months trying it out. There was no day-one benefit, but over those months we managed to wrangle things around until everyone was on the same page. After that, things went a lot smoother. Management was actually planning things out (instead of just throwing projects around and hoping) and we had a clear view of what we would be working on at any given time. Shared resources, like the sysadmin and network admin and db admin, were able to be scheduled more effectively instead of project waiting on them, or them sitting around doing nothing.

So no, I don't think that Scrum requires the kind of buy-in that indicates you don't need Scrum in the first place.

[+] adrianhoward|13 years ago|reply
No, there isn't. You adhere to your burn-down, not to your feature set. Scrum is time driven, not task driven. The whole idea is to become better at estimation so that Scrum appears task driven, when really it's just because your team is that good at estimating.

Erm. There is an explicit commitment by the Scrum team to complete the stories selected to produce the next increment in the sprint. To quote from http://www.scrum.org/Portals/0/Documents/Scrum%20Guides/Scru...

"The two parts of the Sprint Planning Meeting answer the following questions, respectively:  What will be delivered in the Increment resulting from the upcoming Sprint?  How will the work needed to deliver the Increment be achieved?"

The commitment on delivering a certain chunk of work has been one of the more controversial elements of Scrum. Many people who say they're doing Scrum don't do this. But it is very definitely part of the Scrum process as defined.

[+] Domenic_S|13 years ago|reply
> > With Scrum, there is an explicit commitment ... on what stories are going to be delivered within the sprint,

> No, there isn't. You adhere to your burn-down, not to your feature set. Scrum is time driven, not task driven.

Of course there is. There is no burn-down, no 'time', without tasks. So while in a pedantic sense you're right that you commit to 'time', time is measured in stories.

[+] tosh|13 years ago|reply
The problem with SCRUM is that it is not agile. That's why we see most of the pragmatic companies adopting a kanban or scrumban approach and they see good results.

I would argue that just having a kanban board during stand-up meetings and "walking the board" instead of interrogation-like status reporting is going to do wonders to team collaboration atmosphere, morale and actual productivity.

You start to see and talk about the flow of value, what's next, what's in the moment ("what can we do right now?). You see context. All of that is missing with SCRUM or only possible with a highly motivated and responsible team that communicates a lot automatically.

I'm not yet saying that kanban is solving the software crisis [1] but it turned around quite a few teams that I've seen and been part of.

DISCLAIMER: I'm working on https://www.blossom.io which is a lightweight kanban board.

1: http://en.wikipedia.org/wiki/Software_crisis

[+] manmal|13 years ago|reply
Can you elaborate on why Scrum is not agile? You are the first I read claiming this.
[+] leothekim|13 years ago|reply
The author is right that, with scrum you end up focusing more on process and not delivery.

There are nice things about scrum, but I think scrum followers are too doctrinaire. It has some well-defined practices and is associated with that agile "manifesto" that you are compelled to buy into if you adopt scrum. Being doctrinaire about anything is guaranteed to distance you from reality - you give project managers a weapon to coerce you into worrying about how to break down your work into a velocity and stories and tasks. Any time you enforce a process like that, you are disengaging yourself from the reality of having to ship something.

[+] vannevar|13 years ago|reply
Scrum is focused almost exclusively on delivery. Every sprint, you should be delivering working features. It's not that hard: every sprint, you commit to a set of stories to finish before the next sprint. Every day you meet briefly to tell everyone how you're progressing and to air out any impediments. That's about it. To me, scrum is stripping process down to the bare minimum you need to be effective.
[+] vannevar|13 years ago|reply
FTA: Scrum forces iterations, forces feedback, forces smaller iterations. These are all good things, which I loved about Scrum.

And yet the author spends most of the article denying that these aspects of scrum are useful at all. Planning sessions are "highly inefficient," a "quick meeting between the architect and the developer" is better. What if someone else has an important piece of information that the dev and the architect don't? Maybe another dev on the team has done something similar in the past, and could have warned that the estimate was low? He won't be in that meeting, to avoid being "bored to tears" with a story he isn't working on.

So how do you know if Scrum isn’t right for you? If it’s hard. If it’s easy, then it will work for you.

Obviously, if something is hard, there can't be any possible benefit to it. It's so much easier to get rid of all that process stuff and just churn out code as fast as you can. What could possibly go wrong?

[+] gutnor|13 years ago|reply
The author has a point. Scrum is costly. Obviously it is more productive to silo specific knowledge to specific people, generally the expert (or more motivated) in that field.

At least it is in the beginning. People leave, experts become expert teams, silos widen and it soon become the good old planning nightmare we have all learned to "love" in enterprise development.

[+] pherk|13 years ago|reply
> What if someone else has an important piece of information that the dev and the architect don't?

That isn't good enough justification. You are too much erring on the side of caution.

[+] at-fates-hands|13 years ago|reply
> Iteration planning meetings are seriously expensive.

I completely agree on this one. I've worked in several corporate environments utilizing Srum and the planning meetings were always a huge waste of time. I would rather light my hair on fire than sit around a bunch of PM's trying to figure out what features to include/exclude.

Also, most of the people (PM's,Dev's,IA's) I talk to always say, "Nobody does Scrum/Agile right anyways." Which made me wonder if there is indeed a right and wrong way to do these.

[+] jt2190|13 years ago|reply
In my experience most teams do planning completely wrong. The goals of the planning meeting are simply:

  1. Do a relative-size estimate the top n stories in the backlog. (Where n is 
    some number slightly larger than the number of stories that usually fit
    in an iteration.)
  2. Pick the stories to complete in the iteration.
That's it.

I often see teams:

  * doing one-by-one story estimation, and debating over how many points to assign.
    (The statistical method completely fails when it's done this way.)
  * getting into long discussions over the spec. (That's between the dev
    and product owner, to be figured out during the iteration.)
  * worrying too much about accurate estimates
  * worrying too much about how much work to take on
[+] vannevar|13 years ago|reply
I would rather light my hair on fire than sit around a bunch of PM's trying to figure out what features to include/exclude.

Me too. Good thing neither activity has anything to do with sprint planning.

[+] skMed|13 years ago|reply
Perhaps it is the environment or Product Owners that are causing this slow down? Product Owners (PMs, etc.) should stack rank items in their backlog before a sprint planning meeting. This should make it fairly easy to allocate items for a sprint and identify features which either need to be 1. clarified, 2. split/scoped down. My sprint planning sessions have typically always gone smoothly as long as everyone came prepared and ready to contribute.
[+] michaelt|13 years ago|reply

  Iteration planning meetings are seriously expensive.
  Group discussion around design, group estimation, group 
  acceptance, all highly inefficient. [...] I remember just 
  getting bored to tears listening to discussions around 
  stories I wasn’t developing on to begin with.
That's quite team size dependent. In a scrum team with 4 people this isn't a problem - but when the team's twice as large it doubles both the amount of work to plan and the hourly cost to assemble the entire team, quadrupling planning costs.

If a team is structured in such a way that a developer knows they won't be working on a story, that seems like a logical line for splitting into two scrum teams.

[+] vpeters25|13 years ago|reply
Every time I come across "why scrum sucks" articles like these I can quickly point the problem: the ScrumMaster.

#1 - Iterations are less efficient than pull-based approaches: A good ScrumMaster keeps an eye on the burndown chart and negotiates with the stakeholders and team to either add or remove tasks from a sprint. My first sprint ever as ScrumMaster, we estimated a 3 months project, we did everything in 2 sprints (1 month)

#2 – Iteration planning meetings were wasteful: You are doing them wrong. A good scrum master keeps everybody focus on one user story at a time and keeps the meeting moving. I use a 3 min stopwatch in my phone. The whole meeting should not take more than 1 hr (I do 30-45 mins estimate new stories, the rest to plan and commit team to next sprint)

#3 – Scrum is highly disruptive in established organizations: A good scrum master servers as a bridge between traditional management and keeps them out of the team's backs. This one is the reason I don't like being ScrumMaster anymore. A good ScrumMaster needs great people skills, I rather write code.

[+] erichocean|13 years ago|reply
So scrum has a single-point-of-failure that organizations, more often than not, see high failure rates with?

And we're still recommending companies adopt this, when the failure rates are both known to be high, and catastrophic when they occur?

[+] cocojumbo123|13 years ago|reply
Actually what the author describes is a natural path of evolution for that team. Scrum (as a process) contains the possibility of changing the scrum rules themselves - although it is strongly recommended one does that only after they get experienced (e.g. they actually manage to get shippable product each iteration).

When the focus is on the process itself and not on delivery there is something rotten in Amsterdam.

Scrum can be a micro-manager's dream (think of the visibility on who is doing what at almost hourly level) - case in which one can end up with focus on the process not on result.

[+] mmanfrin|13 years ago|reply
Minor counterpoint to the bit about meetings being boring: As a QA person, it is incredibly valuable to us to all be together so we can discuss things with the devs that are otherwise getting overlooked in bugtracking software or even in our chat. It makes it so that those orphaned problems can find a home.

But our meetings are short. 5 or 10 minutes max. If you're getting bored than, to quote another commenter here, you're doing it wrong. Quick summaries, major problems, then get out the door.

[+] base698|13 years ago|reply
You haven't really jumped the shark with scrum until you have in place a scrum of scrums, and a scrum of scrum of scrums taking up half the day for all the key players.
[+] dpham|13 years ago|reply
I work at Webs and we also recently abandoned scrum for a more kanban approach. While all points in the article resonated with our company, the biggest problem we had was not delivering projects on time.

We focused too much on fitting as much work into a sprint as possible for maximized throughput. The problem with that, especially when you have alot of projects going on at the same time, is that alot of work was being burned across the board, but the needle for each individual projects didn't move forward fast enough and we ended up missing deadlines.

Another problem with scrum is that it turns creative developers into code monkeys, and this in turn lowers code quality. Developers are constantly worried about trying to meet deadlines for the next two weeks rather than taking the time to do things correctly. This ultimately creates technical debts and hurts the team in the end.

(also, if you're a manager and you use the developer points burned in order to rate performance and distribute bonus, then fuck you)

[+] pherk|13 years ago|reply
I very much agree with the last point. Objective evaluation of a developer is much more than burned down points.

I worked at Amazon and could see evidently that Scrum was turning good developers into mediocre ones. But not many raised a finger against it as Scrum was seen as the norm. And there was no scientific way to establish this fact.

[+] ajsharp|13 years ago|reply
In my experience, Scrum / Agile / XP tend to be more about the process than the results. I find these methodologies to be particularly useful for contractors and consultants, primarily because the value proposition for a consultant is much different than it is for an employee.

A consultant comes in and typically some sort of statement of work or master services agreement is agreed to by both parties, which outlines roughly the work that will be performed over the course of the engagement. Once work begins, some Agile hoops will be jumped through (storyboarding, etc) in order to further establish and agree upon the work that will be performed. This way, at the end of the engagement, when the consultant has been paid for 3/6/9 months of work, they can point back to what was agreed upon each step of the way and say, "See, we're delivering what was agreed upon way back when."

Employees at most software companies, and certainly at early-staging companies, don't work like this, nor should they. As a product engineer, your job is, in a nutshell, to figure out through software how to make the business work. There might be a high-level strategy laid out for you, there might not be. Either way, you're engaged in an inherently creative activity in order to devise and implement a solution to make users happy, and hopefully make money. Whether or not the employee delivered upon what was agreed to three months ago, or whatever the timeframe, is irrelevant (at least it should be).

The question should be, "Is your work having a positive impact on the business?". There is an inherent necessity to quantify "positive impact" when working with consultants, partly because they cost a lot more in the short-term, and partly because the nature of their work is generally less creative and more controlled.

So, if you trust the people you work with, and the company communicates it's vision well, IMO Scrum and Agile are too much procedural overhead. Of course, you have to meet those two conditions :)

[+] gstober|13 years ago|reply
I read through the first bunch of these comments and scanned the rest...you guys, you guys... I might have missed it but I didn't see anything about client delivery that is viable...or do you have other reasons for being in the software game? Who really cares what process you use? As a client I want to be involved, I want to see how you're doing on an on-going basis, I want stuff that works well, I want stuff I can use and I want to alter by vision as you deliver because my business can change...
[+] snarfy|13 years ago|reply
The effectiveness of scrum really depends on the team size. The larger the team, the more bureaucracy is needed to manage it.
[+] billswift|13 years ago|reply
The larger the team, the more overhead is needed, no matter what system you are using. The real problem is that scrum doesn't scale as efficiently as more structured management systems, so the overhead grows much faster than in most other systems as team size increases.

ADDED: Conversely, with small teams, the overhead of more formal methods doesn't scale down; that is where less structured methods like scrum are most efficient.

[+] genwin|13 years ago|reply
True. My last project started each day with 2 hours of scrum.
[+] ronnier|13 years ago|reply
What's a good team size?
[+] cpenner461|13 years ago|reply
Several comments about the notion that at some point you outgrow Scrum. For those who have reached this point, I'm curious what you've switched to? Or has it been more of simply relaxing some of the constraints/procedures? Asking because I've recently started to wonder if/when we'll need to modify our mostly-Scrum process.
[+] adrianhoward|13 years ago|reply
The most common (productive) change I've seen is a move to something that's closer to the pull/kanban/lean approaches.

For example a pattern I've seen a few times goes something like this:

* Teams get good at breaking down and estimating backlog items, they tend to all become the same "size" and need about the same amount of "work"

* Product Owners get good at prioritising the work so that the most valuable stories are at the top of the queue

* Dev team and product owner turn have a collaborative relationship rather than a boss-minions relationship.

* The combination of those three factors means that the planning meeting basically turns into "we do the next N stories" where N is the number of stories you get done every sprint. So the utility of this meeting disappears.

* The process of work flowing through the team starts resembling single-piece flow. Everybody works on the next most important thing.

* People start continually delivering, rather that just being able to deliver at the end of the sprint.

* The last two points mean the utility of breaking work into sprints kind of vanishes.

* Everybody notices that they're doing flow-ish things rather than batch sprint-ish things and starts looking to the lean / kanban world for ways to optimise the process further.

[+] gleiva|13 years ago|reply
We started doing Scrum around a year ago and it has worked well for us. I feel some of the points described in the article about Scrum are not because of the framework itself but because there could have been a lack of a Scrum Master, which is an important piece in the puzzle, to ensure Scrum effectiveness.
[+] demian|13 years ago|reply
We need to drop the idea that there is THE methodology to organize software development. I like the approach of this post, but I would had prefered a little more context on the type of projects he manages.

A professional software manager should be able to identify the key aspects of a project, and with that define the methodology (borrowing from different aproaches and using different tools). Some projects require centralized design with a top-down approach, and more of a "programming workforce" self-coordinated with some progress metric. Other projects require more sophisticated engineering design, technical expertise imbued in a creative enviroment by a relatively small team of engineers/developers.

In my opinion, iteration is all about exploration. Scrum basically discards the role of a "designer" for the systems. But sometimes you need a designer.

[+] michaelt|13 years ago|reply

  Iteration planning meetings are seriously expensive.
  Group discussion around design, group estimation, group 
  acceptance, all highly inefficient. [...] I remember just 
  getting bored to tears listening to discussions around 
  stories I wasn’t developing on to begin with.
That's quite team size dependent. In a scrum team with 4 people this isn't a problem - but when the team's twice as large it doubles both the amount of work to plan and the hourly cost to assemble the entire team, quadrupling planning costs.

If a team is structured in such a way that a developer knows they won't be working on a story, that seems like a logical line for splitting into two scrum teams.