top | item 20976397

Test-driven development: A great idea hiding behind a terrible implementation?

149 points| kiyanwang | 6 years ago |itnext.io

169 comments

order
[+] Illniyar|6 years ago|reply
He's right, but also wrong. It really depends on what you are developing.

TDD is awesome when fixing bugs (as opposed to writing features), when writing straight up business logic (especially when there are a lot of edge cases) and when you know exactly how something is going to work.

It's really bad when doing anything that requires exploration and research (which is the example in the article).

[+] moksly|6 years ago|reply
I work for a Danish municipality and we buy quite a lot of development from various software houses. Being the public sector we track and benchmark almost everything, and we actually have a dataset on automated testing that’s been running for two decades.

It’s hard to use the data, because we’re comparing different projects, teams and suppliers but our data shows no advantage in choosing the companies that are very test-focused.

They are often slower, more expensive but have the same amount of incident reports as the companies which tests less or doesn’t do automated test at all.

[+] simonh|6 years ago|reply
One thing to consider I think is that TDD isn't just about the tests. It's also about architecting the software into small functions and lightweight classes, which are easy to write concise tests for and then implement incrementally. To TDD advocates, this resulting characteristic of the code is a valuable goal in itself, even just as a side effect of TDD.
[+] Gehinnn|6 years ago|reply
> It's really bad when doing anything that requires exploration and research

I wouldn't say that, I have an counterexample ([1]): It is clear what the refactoring (that is to be implemented) should do and how to state that ([2]), but it usually requires exploration and research how to work with the AST and which methods to use.

I would even say that TDD is awesome for exploring, researching, for getting a feeling of the problem's context, what to do and what the caveats are, as long as neither of that is required for stating the test itself.

Having tests first, you can simply start the debugger, try things out and with hot reloading you can write the code while running the specific test that you want to concentrate on. Visualization tools like [3] can greatly help exploring how to solve and implement the current test while you debug it.

[1] https://blog.hediet.de/post/implementing_typescript_refactor... [2] https://github.com/hediet/hediet-ts-refactoring-lsp/blob/mas... [3] https://marketplace.visualstudio.com/items?itemName=hediet.d...

[+] dnautics|6 years ago|reply
I don't typically tdd, but I'll say TDD is also awesome when you are in a crunch and working on some feature that has, say more than one dependency and more than say four corner cases and you are all amuddle about which one you should do first. Especially, london-style? Is it? Where you write out your trivial stub implementation first, helps me sort it out and not worry about thinking of everything.
[+] simion314|6 years ago|reply
He said that too, and sometimes is better to write the tests after you have the functionality finished and working because you know exactly what you expect and you add the tests to prevent feature updates to cause unexpected changes.

IMO TDD works if your input and output is clear from the start and the way to get the output is convoluted enough that you need the test.2

[+] AstralStorm|6 years ago|reply
It is still useful, just a different kind of test.

High level quality/performance check, rather than testing functions. Produces values, not pass fail. Not really a unit test, an automated integration test of a module.

Simple test run to detect crashes or utter failures in CI fashion helps with speed of development too. Also not a unit test.

Once we had a research solution and were setting up performance improvements, we have put regression tests in place for the replaced functions and the whole system, derived from earlier module quality tests. Added a few well chosen property based unit tests.

This has caught edge cases in the fast implementations.

[+] dgb23|6 years ago|reply
Very well put. It is a great paradigm for problems that you understand very well.

But I think we can also define it more inclusive and still be fine: TDD is a good paradigm for programming against well defined APIs.

For example if you are handed an API specification then locking down your interface(s) with tests in advance can be very helpful.

What is even more helpful is to have a higher level language or abstraction that encodes the specification in detail and generates the tests for you (Clojure spec comes to mind).

edit: typo

[+] cryptica|6 years ago|reply
TDD can work if the tests are integration tests and each test case covers an entire feature (one which can be clearly described using non-technical language). Integration tests only lock down the high level interfaces and behaviors so they allow you to keep reworking the guts of the system without ever having to rewrite the test cases. Unit tests are mostly only good in the very final stages of the project when you have a fully working, essentially perfect, bug-free system and you just want to lock it all down so that nobody can make even the tiniest changes to the code without breaking all the tests and becoming insecure about their abilities in the face of such a remarkable code base. If you're writing unit tests for poorly architected code, you're essentially locking down this horrible architecture and forcing all future developers to find complex ways to work around it instead of allowing them to confidently make the necessary restructurings to allow them to cleanly build on top (this kind of "I'm not breaking any features" confidence can only come from integration tests).

When it comes to complex systems, writing unit tests too early in the project is a bad idea; especially when combined with an agile development methodology. It's like building a skyscraper over several years and perfectly analysing every steel rod, every brick and every pane of glass as it is added to the structure but never actually stepping back to check that the structure as a whole is actually pointing upwards.

[+] tomxor|6 years ago|reply
> He's right, but also wrong. It really depends on what you are developing.

The more useful thing to notice beyond TDD is how this is true of almost every development model and programming pattern (including the patterns built into languages that pretend they are not) - development processes and programming patterns are almost never generally applicable - The problems come when people who don't understand this attempt to apply them generally and blindly, e.g the classic go a read a patterns book or development strategy book and now everything is a nail for their preferred choice of hammer.

I'm not being defensive of TDD though, just because it's subjectively useful does not make it equal, to me TDD looks to have very narrow utility in general for the vast majority of projects it's going to get in the way.

[+] purple_ducks|6 years ago|reply
Great to see the author tackle such a broad topic after 3 months of bootcamp and 1? year of professional development.

Regurgitating others' opinions and making multiple unfounded assertive statements is key to demonstrating he has already mastered the software domain.

[+] scribu|6 years ago|reply
I don't think that's a fair assessment.

He talked to a bunch of experienced devs and summarised their opinions. And then he shared his own opinion on the matter.

Nothing wrong with that.

[+] PUSH_AX|6 years ago|reply
Care to attack the arguments next?
[+] humble_engineer|6 years ago|reply
Its always like this. Last month the article was about how OOP was useless and a waste of billions of dollars, yet the author had like 2 years of experience as a Php developer (in 2019?). It didn't stop 2 of my sub par coworkers from sharing the article and acting like the author had any creds or experience at all to be saying such things. Generally I never read blog posts from no name people claiming industry standards should be destroyed. Its just a pointless exercise.
[+] djrobstep|6 years ago|reply
The experiences of beginners are as valid as anybody else's.

It creates an unfriendly culture if people are told they can't talk about their frustrations and challenges without having x years of experience first.

[+] GoodGOYIM|6 years ago|reply
I find it amusing to read posts like the authors. I bet he also does not even use Agile Methodology lol ;-)
[+] Gehinnn|6 years ago|reply
There clearly are problems that are suited very well for TDD and others that aren't.

I guess the suitability of TDD depends on how much a feature involves "how" rather than "what": Whenever it is clear what the feature should do (and what the feature's interface looks like) and the implementation is only about the "how", TDD seems to a very good idea. Almost every compiler-related task falls into this category. I just wrote a blog article on how to implement Typescript refactorings using TDD ([1]) and I don't think anyone could argue against using TDD there.

If the feature is more about "what" (like UI/UX), TDD seems to be more of a burden than an useful asset.

[1] https://blog.hediet.de/post/implementing_typescript_refactor...

[+] qznc|6 years ago|reply
One important aspect of tests is to derive them from requirements. Not from the implementation. This is also true if your requirements are only implicit in your head.

TDD ensures this as there is no implementation yet when you write the tests.

It also implies the changing requirements comes with the overhead of changing the corresponding tests. This is what happened in the incident described in the article. In many cases development speed might be more important than test coverage.

Still worse are tests derived from the implementation (usually to boost done coverage metric) because they often break although the code fulfills the requirements.

Rule of thumb: Test only the requirements which are stable.

[+] xthestreams|6 years ago|reply
Regardless of my opinion on the topic, which I don't have:

why must software engineering be mainly opinion-based? Why can't we have objective facts, or at least only talk about objective facts without going out asserting one's opinions as truth?

AFAIK other engineering disciplines do not suffer from this problem as much as software engineering. This really bugs me.

[+] BigJono|6 years ago|reply
Because software projects fail for people reasons, and people reasons are usually subjective.

If you have a tight team of experienced devs building a simple CRUD app you can do almost anything short of building the thing in Polish BASIC with triple spaced lines, and you'll still get a result. Likewise if you have a team with 8 managers and your best dev has only done a couple of courses on codecademy, no set of engineering approaches is going to save you.

And that's just the factors within the engineering team. For every inexperienced team that doesn't deliver a result, there's 10 teams that are gutted from internal politics or management decisions, and 100 startups that fail to find a market fit. On the flip side of that, every month or so I find either a showstopping bug or a new puzzler of a UX decision in one of Uber's products, but it literally has no bearing on their success as a company. The simple fact is it doesn't matter whether they do TDD, or scrum, or whatever, as long as they don't do anything truly catastrophic.

Basically, the majority of success factors for the majority of projects come from outside the engineering team. And the ones that don't will be dominated by the team's basic development skills, which makes the signal much worse for measuring the effectiveness of anything from a project management, high level architecture, or testing perspective.

[+] tadzik_|6 years ago|reply
Well call me crazy, but imho the reason for that is that for the most part, software engineering is not engineering. We're not fighting laws of nature, we're fighting other people's style and decisions. That's why our work is opinion based: we're basically computer lawyers.
[+] nemetroid|6 years ago|reply
I think the question is easier to answer if it's flipped. Why can't other engineering disciples work fast-and-loose?

The answer is that for software engineering, the cost balance between "working fast" and "avoiding errors" largely works out in favour of "working fast", even with the potential error rates factored in. In other engineering disciplines, the cost of failure is higher (often much higher), and therefore "slow and steady, avoiding errors" methodologies win out in this balance.

[+] bastih|6 years ago|reply
There is a good amount of research on software engineering, I love love love "Making Software: What Really Works, and Why We Believe It" by Andy Oram, as it demonstrates the non-opinion-based way of thinking about software engineering. And has some really interesting insights to boot.
[+] otabdeveloper4|6 years ago|reply
Two points.

First, software "engineering" is not an engineering discipline. You might as well call novel writing "literary engineering", but calling it that won't make it true.

Second, the 10x software developer is very much real. People try to ascribe their performance to this or that technology or methodology, but in reality what makes a 10x engineer is a special psychological predilection, not their tools.

[+] Retric|6 years ago|reply
Because this is really project management not software engineering. You would get into real arguments about how to micro manage a team of engineers designing a space shuttle replacement. Complex expensive tasks with real tradeoffs don’t have simple answers.
[+] kissgyorgy|6 years ago|reply
As I have more and more experience in the software development field, the more I think it is not engineering, but rather art. At least it's a creative process, not always an exact science.
[+] fagnerbrack|6 years ago|reply
That's because there are too many developers with the title of senior but skills of junior, and a business person can't tell which one is which unless they're one of the few high experienced developers out there.

The only time where a business person knows they hired the wrong people is when people claim for a rewrite. At that point, sorry, it's too late.

[+] johnmarcus|6 years ago|reply
Probably because we are dancers, after all.
[+] gtirloni|6 years ago|reply
Not much to add except that when j tried TDD, I had to have this mentality of typing out code as fast as I could, going back to the test, fixing stuff, back to code, learn the mistakes, etc.

While my dev approach is more about think, think, think harder, code, compiler barely complains, fix small issues, write test, done.

TDD felt very uncomfortable because the thinking part didn't seem to be emphasized much. It's like keep going, you iterate again real soon in this code, now again, and again.

Maybe I got it wrong, I don't know. But I share the author's frustration.

[+] johnmarcus|6 years ago|reply
I'm with you and the author on this.
[+] DanielBMarkham|6 years ago|reply
"...I never understood why it was better to do things backwards...The idea behind TDD is that you’re able to write a test because you sit down and think about things first...."

Yeah no. You have been instructed poorly. It's not your fault.

Yes, you have to think about things, but that's not the real purpose of TDD. Just like making a good design, it's a side effect.

Straight coding is doing it backwards, as odd as that sounds. The reason is that the computer never lies, but it's really, really easy to lie to yourself, forget things, misunderstand your own variable names, and so on. When you're coding, you'll make all kinds of errors. The computer will not. So wouldn't it be better to have the computer constantly checking on you to see if you're making any new errors? Better still, not only will it check on you, it'll check on everybody else who touches the code too.

There are domains in which TDD does not make sense. Exploratory coding, like you'd do in startups, for instance. Small pure functional code. Academic learning. In each of these cases you are still doing TDD, it's just you're not coding the tests. You never get away from TDD, you just sometimes hide it or pretend it isn't happening. There's a test, it fails, you write code to make it pass.

In other cases, such as any non-trivial mutable OO commercial code that is going to be around for a while, not using TDD is malpractice. Whatever small hit you may take on initial productivity is insignificant compared to the endless amount of money you may spend later on maintenance.

Plus, I'm not so sure it's inefficient to begin with. Instead, I think a lot of coders code in an inefficient way writing code that is difficult or impossible to test. Once you start using TDD, you're going to have to come to terms with that and un-fuck your code. That's probably going to suck. Best to do it earlier rather than later. (Long discussion here about what to do with legacy codebases)

[+] eterm|6 years ago|reply
I'm generally inclined to agree, test coverage is good, but test driven development is awkward. I think there's a tinkerer's mindset which is very much stifled by TDD.

I think TDD does shine for some things, it especially shines for refactors (where the tests ideally shouldn't change anyway) or bug-fixing where you can write in a test to catch the bug that has been found as a regression test before starting to fix the bug.

For brand new features it is a signifincant discomfort. I concede that that discomfort can result in more thought-through code but it can also just put off the implementer into not working as efficiently as they would be able to in a 'code first, then refactor' mindset.

[+] lolinder|6 years ago|reply
On the flip side, "code first, then refactor" is only safe if you have tests to confirm the validity of the refactor, and in my experience, a TDD mindset is the only thing that ensures I have thorough tests by the time I need to refactor.

That said, it sounds like my version of TDD may be different than a lot of others. The tests I write before coding tend to test real-world logic and requirements, in terms of the domain language, rather than testing implementation. This means I can refactor everything behind that DSL-like facade and my tests give me assurance that I haven't changed the logic.

If your tests are at the implementation level, I can see how a TDD approach would impede refactoring and eliminate tinkering. If refactoring the code means rewriting the tests, then your tests aren't protecting you, they're just getting in the way.

[+] polytronic|6 years ago|reply
I concur! Additionally to the article I believe the overall problem with software development is that it cannot be done by everyone. Developing great software requires the mentality of a passionate artist and a lifetime dedication. Unfortunately most companies I've worked with (with very few exceptions) are only after immediate profit and thus creating toy programs mainly by means of integration of ready-made third party libraries. The example they set for new developers is catastrophic. Fewer and fewer people dwelve into the depths of knowledge. Shallow integration, adoption of over-hyped immature components and the new trend of containerization is a two-fold failure: a) Failure of educating minds to think and b) low quality end products for the users with all the consequences this brings (aka performance, security, UX, etc). It really feels as if software development has entered a dark era where instead of educating people we create mechanics that take 'experinments' to market asap and disguising them to look like they operate error-free. Of course such tactics cannot keep problems hidden forever. If only I could share with you the stories of large heap fragmentation I've heard over the years on production server code...
[+] bluesign|6 years ago|reply
This is skipping the purpose of TDD actually, TDD is basically binding your every code change to a test (basically it is a spec) You are supposed to write the minimum required code to satisfy a failing test. Basically it is kind of commenting every bit of your code with a test.

So every line of code you are writing has to have a purpose, and later on when you are fixing a bug or adding something new, you are making sure, you didn’t break any previous functionality.

[+] to11mtm|6 years ago|reply
And yet when an oracle dev comments on HN about the test suite we all gasp in horror...
[+] investologia|6 years ago|reply
This comment in the original article is really to the point:

I think one of the biggest values of TDD is it teaches developers how to write unit testable code, which, if you are new to unit testing, can be quite hard to get your head around.

As for its usefulness in ‘real-life’, I’m going to answer with a solid ‘it depends’.

Anyone who dismisses it is missing out on some potential big advantages, anyone who is a 100% TDD zealot is probably either wasting time or lacks extensive real world experiance.

[+] swiftcoder|6 years ago|reply
I tend to think that the biggest flaw with TDD is that it assumes you have requirements, from which to derive test cases.

And In 7+ years of engineering at FAANG companies, I've never once had clear requirements before the software was complete (let alone started).

[+] sorokod|6 years ago|reply
One claim TDD makes is that a reasonable architecture arises from from its basic approach. To me this is an extraordinary statement of which I have seen no proof.
[+] twic|6 years ago|reply
TDD doesn't make that claim, because TDD is not a thing which can make claims. Some people make that claim about TDD. Those people are indeed mistaken.

As someone who thinks TDD is useful, i find those people particularly infuritating, because such an obviously silly and easily disproven claim turns the programming public off TDD for no good reason.

[+] je42|6 years ago|reply
A couple of things to mind when talking about TDD.

1. Tests can be unit-test, integration-test, e2e tests. Don't limit yourself.

2. TDD is a technique. Depending on the problem and your experience it might not be the right one to efficiently get you a working program and a good test suite.

Remember there is no free lunch.

3. Also, if you have never successfully applied TDD: Please stand stil for a moment. Some techniques require practice and TDD is one of those.

You need to invest time in order to use it efficiently. Once you do it, you will be faster and deliver higher quality code for a large set of programming tasks.

4. There are tasks where i don't use TDD. However, I usually go back to it as soon as I can. You can switch between the two techniques. There is not one way that is most correct.

5. Also, note: if you don't write tests first. Writing tests after the code is complete is more difficult. You might need to adapt your code to make it testable.

6. If you write tests afterwards, usually you will accidentally omit tests for crucial cases if you are not careful. I use line coverage and branch coverage tools to help me when I don't develop test driven.

[+] gilbetron|6 years ago|reply
Testability, Performance, Readability, Maintainability are all vectors we can judge software by, and while they are not orthogonal, they are also rarely colinear. TDD optimizes on Testability to the expense of the others. This is rarely a good idea.
[+] jomkr|6 years ago|reply
Development is the ability to deliver/ship the things that increase your business metrics, consistently.

Sometimes the right thing to do is TDD, sometimes the right thing to do is to ssh into a production server and modify a Perl script.

[+] tegiddrone|6 years ago|reply
It's painful when I come to a project and the process for developing a feature is...

  (1) install database (or point to dev,) configure app, run seed data. 
  (2) creating the state in the database manually. 
  (3) change lines of code for feature
  (4) reloading app, navigating through to the feature state, manually verifying. 
  (5) fix runtime errors or bugs
  (6) Repeat as needed. Commit.
We don't need to swing around the entire stack to craft some pixels or make sure some message is displayed under specific circumstances. How do we get developers to think beyond this workflow?

I think TDD at the very least is a education/comprehension exercise in abstract thinking which will discourage scenarios like the one described above. Once someone is able to TDD and 'gets it' then sometimes the way they approach arranging code even without TDD is better. I think this is why we may have mixed messages like "TDD is great!.. but we don't practice it at work."

Using a test harness any sorts to test something in isolation will speed things up. We can better understand where a module begins and ends (i/o, side-effects.) Develop it rapidly in the harness and then a few (automated) integration tests at the end. It doesn't have to be a TDD-specific testing harness.

React Storybook is an example of a UI test harness. It's still manual testing, but we get that great separation-of-concerns goodness that makes for better designed (and easier to maintain) systems.

[+] honkycat|6 years ago|reply
TDD is so tragically misunderstood. I don't think this dude knows what he is talking about.

Case in point: you don't have to write your tests first. You can write a quick spike first to feel out you API, then rewrite that code with TDD.

Why is this dude so desperate to have this function written quickly? This whole episode could not have taken more than 20 minutes. Sloppiness is the enemy with software development.

"But in real life, features are way more complex than Function X should take a name and output a greeting with that name. Often times the product request will change mid-work, or you’ll realize that the feature can’t work as requested. Maybe your original understanding was flawed and you have to start over."

Smoking gun right here. This is just wrong and stinks of amateur. Features are almost always made of many small functions that do little bits of easily tested work. And if you factor you code well, which is a large benefit of TDD: having well factored code, changes should not collapse you whole sand castle. If that is happening, maybe go back to school. Oh wait, you never went to school, just a 4 week course so you can have low level programming job to toil at...

What he is missing here is the alternative: a big ball of mud with giant functions and poor tests to validate behavior.

[+] fagnerbrack|6 years ago|reply
TDD is one tool in your toolbox. You can easily solve problems like this one: https://itnext.io/you-dont-know-tdd-691efe670094.

However, there's a myriad of other problems TDD doesn't help you, like when you need to design a system beforehand or you want to make an educated jump to a solution that's good enough and you already know it.

[+] maaaats|6 years ago|reply
I have learned and used TDD before, but I don't do TDD. I am, however glad I learned it, as it influences the way I write code now to the better. (Think more before I start writing, for instance, and writing smaller testable units of code up front). Much in the same way learning a new programming language/paradigm also can change how you write code in the old language.