This is an interesting article but even more interesting is the series of articles that the author links to. I've been involved in several high profile products with different locations on the too much or too little debt axis. It always felt like economics could bring a more level headed viewpoint to the discussion instead of just relying on cargo cult mantras. I've skimmed and read the articles and they look like a good intro to the discussion.
As an example, I've found that most successful project have an extremely uneven distribution of debt with most of the code being glued together with bubblegum but a few critical regions having the polish of a high quality gem. Problematic projects tend to have an even quality with all parts having an equal level of 'cleanliness'. This article has some good discussion on why maintaining a similar level of quality across the system might not be a good idea:
http://www.alphaitjournal.com/2009/06/cross-discovering-asse...
Technical Debt at a startup will be always be present at a large scale as long as your main mission statement is getting to market as soon as possible. Their first question is 'does the market want this' instead of 'will it scale when it reaches mass numbers'?
I always consider Facebook as a prime example. The first version may very possibly have been something hacked together in PHP, with the intent of releasing something asap that had a good chance of flourishing. Now that it has flourished, and the proof of concept is validated (with VC money), you can afford engineers who will rewrite the system the way it was meant to be, complete with a PHP compilers and custom flat file databases. The risk paid off.
Digg's technical debt served them well too. They got into the market fast, and the performance issues of the previous site wasn't nearly as big as a factor as the business domain problems they introduced. Submitters didn't mind the occasional 'invalid token' error messages nearly as much as the revolt caused by the publish of a certain encryption code.
Joel says don't rewrite from scratch. We talk about big design up front, when the design doesn't even appeal to the market. But when your product has reached a critical mass, and you have an army of talented engineers salivating to rewrite the system, all that debt is forgiven when the system is redesigned from scratch.
What about Netscape? They did a re-write after their product reached a critical mass no? Or maybe I'm wrong (hence, Netscape is not THE most sought after product)?
In my opinion, having a reasonably clean architecture is all that matters.
Not that bad: the implementation of an individual function is counterintuitive and undocumented--as long as an intuitive implementation of the function would interface with the rest of your software just the same way.
Actually pretty bad: the function has side effects that would be better off somewhere else. If this is the case, fixing this function is likely to break something, which may break something else, and so-on.
The ideal case is one where even though your code base is messy, you can easily see how you could achieve a clean code base through a series of short, discrete steps, where each step begins and ends with working code.
Kind of like how if you've got a string attached to something at the entrance of a cave, it doesn't matter much how deep in to the cave you get--all you have to do is follow your string back.
Of course I'm just a 19-year-old kid. Someone who's worked on a large software project should tell me if they like my theory or not.
Of course I'm just a 19-year-old kid. Someone who's worked on a large software project should tell me if they like my theory or not.
Your post would be just fine if you left that line off. It was well-stated, agrees reasonably with my experience, and provides concrete points that can easily be addressed by anyone who disagrees.
My next suggestion would be to try it yourself: join a large software project.
Each developer would tell different stories. I thought I'd tell mine a bit. In my office, we're embarking a series of cleaning up the code because it's a big pile of band-aids. People were told to read "Working Effectively With Legacy Code".
While the book has been godsend, people's mentality is a different issue. Developers are an interesting bunch; they complain a lot when the code is full of band-aids but they will also back-off when management give a full-blessing to start cleaning up pieces by pieces. The only motivated people to do the task are mostly the new hire.
The software is a typical client-server that utilizes a standard/minimum WebService (WS is not as bad as you thought it would be). But here's the weird part: the server is a dumb component. Let me explain via ascii art:
Client(GUI, heavy BiznezLogic, heavy Validation) -> WebService (dumb) -> Data Access Layer (dumb) -> Stored Procedure (complex logic and validation).
Sort of good separation isn't it? Except it's hard to write unit/integration tests around heavy GUI based software and around Store Procedures. (Of course one could still write integration tests on top of the DAL layer, but we'll never achieve unit-test since we can't test per Stored Procedure).
The thing is: we can't drop everything completely and start refactoring (even if it's just a component). The train must continue.
I used to dislike consulting, but these days I've been starting to look at that direction. An added benefit of consulting is that you can meet clients and start networking: something that a maintenance developer can't do, cause he's stuck either fixing bugs or refactoring code until late night (and adding more unit-tests).
Unfortunately, most jobs are maintaining other people's code unless you're working for a startup or your own project. When you work in a startup, quality usually get pushed. Eventually when you hit year 2.5, you will deal with the same issue again and again. This vicious cycle keeps repeating over and over.
PS: Don't get me wrong, I love the whole idea of automated testing (unit and integration). I think developer should be accountable for the mess they created. I also think the business people should sit next to the developer once in a while.
[+] [-] kaib|15 years ago|reply
As an example, I've found that most successful project have an extremely uneven distribution of debt with most of the code being glued together with bubblegum but a few critical regions having the polish of a high quality gem. Problematic projects tend to have an even quality with all parts having an equal level of 'cleanliness'. This article has some good discussion on why maintaining a similar level of quality across the system might not be a good idea: http://www.alphaitjournal.com/2009/06/cross-discovering-asse...
[+] [-] gigantor|15 years ago|reply
I always consider Facebook as a prime example. The first version may very possibly have been something hacked together in PHP, with the intent of releasing something asap that had a good chance of flourishing. Now that it has flourished, and the proof of concept is validated (with VC money), you can afford engineers who will rewrite the system the way it was meant to be, complete with a PHP compilers and custom flat file databases. The risk paid off.
Digg's technical debt served them well too. They got into the market fast, and the performance issues of the previous site wasn't nearly as big as a factor as the business domain problems they introduced. Submitters didn't mind the occasional 'invalid token' error messages nearly as much as the revolt caused by the publish of a certain encryption code.
Joel says don't rewrite from scratch. We talk about big design up front, when the design doesn't even appeal to the market. But when your product has reached a critical mass, and you have an army of talented engineers salivating to rewrite the system, all that debt is forgiven when the system is redesigned from scratch.
[+] [-] hello_moto|15 years ago|reply
[+] [-] astrofinch|15 years ago|reply
Not that bad: the implementation of an individual function is counterintuitive and undocumented--as long as an intuitive implementation of the function would interface with the rest of your software just the same way.
Actually pretty bad: the function has side effects that would be better off somewhere else. If this is the case, fixing this function is likely to break something, which may break something else, and so-on.
The ideal case is one where even though your code base is messy, you can easily see how you could achieve a clean code base through a series of short, discrete steps, where each step begins and ends with working code.
Kind of like how if you've got a string attached to something at the entrance of a cave, it doesn't matter much how deep in to the cave you get--all you have to do is follow your string back.
Of course I'm just a 19-year-old kid. Someone who's worked on a large software project should tell me if they like my theory or not.
[+] [-] nitrogen|15 years ago|reply
Your post would be just fine if you left that line off. It was well-stated, agrees reasonably with my experience, and provides concrete points that can easily be addressed by anyone who disagrees.
My next suggestion would be to try it yourself: join a large software project.
[+] [-] hello_moto|15 years ago|reply
While the book has been godsend, people's mentality is a different issue. Developers are an interesting bunch; they complain a lot when the code is full of band-aids but they will also back-off when management give a full-blessing to start cleaning up pieces by pieces. The only motivated people to do the task are mostly the new hire.
The software is a typical client-server that utilizes a standard/minimum WebService (WS is not as bad as you thought it would be). But here's the weird part: the server is a dumb component. Let me explain via ascii art:
Client(GUI, heavy BiznezLogic, heavy Validation) -> WebService (dumb) -> Data Access Layer (dumb) -> Stored Procedure (complex logic and validation).
Sort of good separation isn't it? Except it's hard to write unit/integration tests around heavy GUI based software and around Store Procedures. (Of course one could still write integration tests on top of the DAL layer, but we'll never achieve unit-test since we can't test per Stored Procedure).
The thing is: we can't drop everything completely and start refactoring (even if it's just a component). The train must continue.
I used to dislike consulting, but these days I've been starting to look at that direction. An added benefit of consulting is that you can meet clients and start networking: something that a maintenance developer can't do, cause he's stuck either fixing bugs or refactoring code until late night (and adding more unit-tests).
Unfortunately, most jobs are maintaining other people's code unless you're working for a startup or your own project. When you work in a startup, quality usually get pushed. Eventually when you hit year 2.5, you will deal with the same issue again and again. This vicious cycle keeps repeating over and over.
PS: Don't get me wrong, I love the whole idea of automated testing (unit and integration). I think developer should be accountable for the mess they created. I also think the business people should sit next to the developer once in a while.