I think TDD is useful, but practitioners should not be dogmatic about it.
As I discussed in my talk "Testing is Overrated", programmer testing finds a different class of bugs than manual testing or code reviews.
One of the problems is that programmers are not very good at writing tests. They are much more likely to test "happy path" scenarios. Steve McConnell reports in Code Complete that mature testing organizations have 5 times more dirty tests than clean tests, while immature testing organizations write 5 clean tests for every dirty test.
Another big problem is that unit tests are never going to show you that your software blows. Only user testing can find that.
If you want to ship good software, I think you need to do a combination of code reviews, unit tests, system tests, user studies, and manual testing. Different organizations will find a balance that works for them.
That last is a key point. I like writing tests, but I hate the dogmatism that if you're not writing tests you're doing it wrong. Obviously, good software has been written without tests, and buggy crap has been fully TDD'ed. In the end, the team matters more than the methodology.
The article misses an important point: People who've done TDD know that it is the easiest/fastest way to unit-test. Few aspects of programming are more painful than trying to unit test code after the fact, especially if it was written without testing in mind.
It's not the fastest way, by far. It tends to lead to more testable code, sure, but it's not fast.
For example, TDD encourages code churn: you write tests for a lot of throwaway code.
Personally, it usually takes me two/three attempts before I'm satisfied with the version of the code that I have. Writing tests for these throwaway attempts is a waste of time.
As you become more experienced in writing tests and testable code, you realize that you no longer need TDD to write code that can be easily tested, so I see TDD more as "training wheels" for beginners.
I fail to see how unit tests are required for a shippable product. For a wide range of things, buggy and shipped is better than clean and ready in 3-6 months.
"If we control the bug rate, is TDD faster?" is the central question. I assert that TDD is the Cause of the bugs, so the question is nonsense.
Code should be developed with as large a view of the design as possible. TDD restricts development (by definition) to satisfying the tests. The effect is of 'moving the bugs around' since the existing tests rarely completely describe the whole design. You get into a loop of "oh, yeah, well we Really meant to write a test to cover that case too", repeated ad-nausuem.
I think one problems with TDD is that it doesn't lean towards testing bugs in a feature/scenario (describing the whole design). That is, TDD focuses on testing a unit and it is effective/fast at testing the fringe cases of units.
Behavior Driven Development really shines in the aspect of "completely describing the design" as it focuses on driving development against a well defined behavior (using scenarios). BDD lets you know where the bug is: in the code or in the specifications (the behavior as specified by the customer). So there is a lot less "chasing around". If you are trying to cover behavior of a system only using TDD then I agree you can end up chasing around a bug.
However, both are necessary and highly effective when used together.
I'm not sure what you mean by "TDD is the Cause of the bugs" - the TDD projects had 40-90% fewer bugs. Even if their remaining bugs would have been caught by test coverage of things that occur in live use, that's still a good result, right?
I would argue that TDD makes you think more about the design of your system. Also, it makes it easier for you to refactor (as you don't worry too much about accidentally breaking something) and hence keep the code quality high over time
"Dependent Types: A Look At The Other Side" by Larry Diehl
It talks about using Types for Driving your Development. So it is TDD, but without tests, without runtime and programmer's overhead of them.
When using TyDD, you isolate core that should go as bugless as possible and write DSeL for it in language with strong type system. The talk above tells about Agda, but we and some number of other companies in the world (Eaton http//eaton.com/, Echo http://aboutecho.com/ - from the top of my head) successfully employ Haskell for that task.
That way you shoot two birds with one stone: you get provably correct software and you get an ideal language to express your core problems.
Best of all, you get free "requirement tracking" - whenever your requirements change, you express them in types and then apply changes where they should, guided by friendly compiler. Still getting provably correct software (modulo types) at the end.
That was "interesting" in that I like expanding my horizons, but it was a terrible presentation. Plus, it looks like Haskell and I got the impression from his lead-in that he was going to introduce a new language.
However, I actually came here to say something else about his material: if it is harder to decompose the business requirement into Haskell than it is to write monkey tests for one's Blub language, I am not certain anyone wins when going with a provably correct implementation.
Anecdote: I'm not smart enough to design software without having an idea of how it can be automatically tested. My design has to prove that it can be easily tested (both via unit tests and integration tests), and I've found that writing some of the tests up front helps with that.
[+] [-] 100k|15 years ago|reply
As I discussed in my talk "Testing is Overrated", programmer testing finds a different class of bugs than manual testing or code reviews.
One of the problems is that programmers are not very good at writing tests. They are much more likely to test "happy path" scenarios. Steve McConnell reports in Code Complete that mature testing organizations have 5 times more dirty tests than clean tests, while immature testing organizations write 5 clean tests for every dirty test.
Another big problem is that unit tests are never going to show you that your software blows. Only user testing can find that.
If you want to ship good software, I think you need to do a combination of code reviews, unit tests, system tests, user studies, and manual testing. Different organizations will find a balance that works for them.
That last is a key point. I like writing tests, but I hate the dogmatism that if you're not writing tests you're doing it wrong. Obviously, good software has been written without tests, and buggy crap has been fully TDD'ed. In the end, the team matters more than the methodology.
http://www.scribd.com/doc/8585284/Testing-is-Overrated-Hando... http://www.scribd.com/doc/8585444/Testing-is-Overrated
[+] [-] Ramone|15 years ago|reply
[+] [-] billmcneale|15 years ago|reply
For example, TDD encourages code churn: you write tests for a lot of throwaway code.
Personally, it usually takes me two/three attempts before I'm satisfied with the version of the code that I have. Writing tests for these throwaway attempts is a waste of time.
As you become more experienced in writing tests and testable code, you realize that you no longer need TDD to write code that can be easily tested, so I see TDD more as "training wheels" for beginners.
[+] [-] Retric|15 years ago|reply
[+] [-] JoeAltmaier|15 years ago|reply
Code should be developed with as large a view of the design as possible. TDD restricts development (by definition) to satisfying the tests. The effect is of 'moving the bugs around' since the existing tests rarely completely describe the whole design. You get into a loop of "oh, yeah, well we Really meant to write a test to cover that case too", repeated ad-nausuem.
[+] [-] ericHosick|15 years ago|reply
Behavior Driven Development really shines in the aspect of "completely describing the design" as it focuses on driving development against a well defined behavior (using scenarios). BDD lets you know where the bug is: in the code or in the specifications (the behavior as specified by the customer). So there is a lot less "chasing around". If you are trying to cover behavior of a system only using TDD then I agree you can end up chasing around a bug.
However, both are necessary and highly effective when used together.
[+] [-] newt|15 years ago|reply
[+] [-] prateekdayal|15 years ago|reply
[+] [-] thesz|15 years ago|reply
"Dependent Types: A Look At The Other Side" by Larry Diehl
It talks about using Types for Driving your Development. So it is TDD, but without tests, without runtime and programmer's overhead of them.
When using TyDD, you isolate core that should go as bugless as possible and write DSeL for it in language with strong type system. The talk above tells about Agda, but we and some number of other companies in the world (Eaton http//eaton.com/, Echo http://aboutecho.com/ - from the top of my head) successfully employ Haskell for that task.
That way you shoot two birds with one stone: you get provably correct software and you get an ideal language to express your core problems.
Best of all, you get free "requirement tracking" - whenever your requirements change, you express them in types and then apply changes where they should, guided by friendly compiler. Still getting provably correct software (modulo types) at the end.
[+] [-] mdaniel|15 years ago|reply
However, I actually came here to say something else about his material: if it is harder to decompose the business requirement into Haskell than it is to write monkey tests for one's Blub language, I am not certain anyone wins when going with a provably correct implementation.
[+] [-] joevandyk|15 years ago|reply
[+] [-] gasull|15 years ago|reply
https://webcache.googleusercontent.com/search?q=cache:grokco...