Is the popularity of unit tests waning?(binstock.blogspot.com)
binstock.blogspot.com
Is the popularity of unit tests waning?
http://binstock.blogspot.com/2008/05/is-popularity-of-unit-tests-waning.html
9 comments
I drank the Kool-Aid and I still like the flavor. I still practice TDD every day. And yet I agree with most of what you said.
When people get dogmatic about unit tests, they write way more than is needed to bring value. The value is: you can refactor code more easily; you can think things through a little bit at a time and let the design evolve; the code you write tends to be really simple and easy to understand; you get executable documentation of code as a side-effect. And for me, a huge benefit is that when pair-programming, the current test defines the current task.
If the unit tests are making the code harder to refactor and evolve, then it's time to write fewer of them. And if unit testing isn't fun, then something is seriously wrong.
Do people not know that when you want to just explore a wild design idea, you should abandon unit testing and just go wild?
When people get dogmatic about unit tests, they write way more than is needed to bring value. The value is: you can refactor code more easily; you can think things through a little bit at a time and let the design evolve; the code you write tends to be really simple and easy to understand; you get executable documentation of code as a side-effect. And for me, a huge benefit is that when pair-programming, the current test defines the current task.
If the unit tests are making the code harder to refactor and evolve, then it's time to write fewer of them. And if unit testing isn't fun, then something is seriously wrong.
Do people not know that when you want to just explore a wild design idea, you should abandon unit testing and just go wild?
I have found out that unit testing is great if you use it sparingly, that is only use it on public methods of public API. So, if I write server-side software, I unit-test only facade methods that are invoked by clients. And I always run the tests before committing new version, which is a pretty good safeguard.
It has saved me some headaches - especially when software grows in size and you simply cannot predict what impact even a simple change can have.
Anyway, I wanted to say that I agree with most of what you said :-)
It has saved me some headaches - especially when software grows in size and you simply cannot predict what impact even a simple change can have.
Anyway, I wanted to say that I agree with most of what you said :-)
I don't see it waning. I do see it dissolving more into general programming practice, as opposed to being a big deal in and of itself. What does seem to be waning is the evangelical fervor surrounding it, which seems like a good thing to me (though not to the author). Like other fads in software, unit testing was oversold.
Meanwhile, the logic of the article seems weak:
1. A company that claimed to auto-generate unit tests for your code failed? Give me a break. You can't write good tests without understanding what you're testing. If it could do that, why didn't it just auto-generate all your code in the first place? One needn't look far to see why this failed.
2. Few OSS tools have been adopted (except xUnit which is used massively)? This is normal. Most open source projects fail, a few succeed big. xUnit won. You don't need much else.
3. Not everyone teaches it? Why should everyone do anything?
4. Not many new unit testing books? There are many such books already. What don't they cover? Why should there be a whole book about JUnit 4?
5. Alternative unit-testing frameworks aren't used... see #2.
The author seems to think of unit testing as if it were a domain. It isn't, it's a technique. Why should there be an industry around it?
Meanwhile, the logic of the article seems weak:
1. A company that claimed to auto-generate unit tests for your code failed? Give me a break. You can't write good tests without understanding what you're testing. If it could do that, why didn't it just auto-generate all your code in the first place? One needn't look far to see why this failed.
2. Few OSS tools have been adopted (except xUnit which is used massively)? This is normal. Most open source projects fail, a few succeed big. xUnit won. You don't need much else.
3. Not everyone teaches it? Why should everyone do anything?
4. Not many new unit testing books? There are many such books already. What don't they cover? Why should there be a whole book about JUnit 4?
5. Alternative unit-testing frameworks aren't used... see #2.
The author seems to think of unit testing as if it were a domain. It isn't, it's a technique. Why should there be an industry around it?
I am not sure that the failure of commercial products, frameworks and books on unit-testing are a sign for the decline of the usage of unit-tests in general. In our company, we are extensively using unit-tests, but use no books neither commercial products (only the junit-framework). Maybe unit-tests have just become common programming practice without the need for any more hype?
I looked into Agitar's JUnit Factory, but had to turn away from it when I realized that you send your source code to them to generate the tests. I have yet to work for a company that would feel comfortable doing that.
We're always looking for a Silver Bullet. Does someone know if there are papers out there that correlate unit testing with better results in software projects?
Even if there were papers on it, I think there are too many confounding factors to elicit anything useful from them. Personally though, unit testing and test first development help me focus on a small problem and solving that, while ignoring the rest of the system. It's really easy to get pulled into the big picture. I also found my code to be more modular and decoupled.
It takes as long to master TDD as it does to master a new language. I didn't get good until I had been doing it for a few years. @ardit33 is dead-on about the coolaid analogy. You either get it and can't imagine programming any other way, or there are significant roadblocks you don't want to overcome (UI) / just can't TDD fast enough to justify the time.
I haven't hit a point where I've invested too much into the quality of the software. It's fine with me if TDD isn't popular. I'll just keep using it as a tool in my arsenal to beat the competition.
I haven't hit a point where I've invested too much into the quality of the software. It's fine with me if TDD isn't popular. I'll just keep using it as a tool in my arsenal to beat the competition.
Unit Testing works great in Smalltalk. Refactoring Browser handily refactors your tests at the same time you're refactoring your code. I've heard complaints from people using other languages that Unit Tests make refactoring twice the work. There is no excuse for this. If your Unit Tests are just code, then the tools that change your code should just as easily change the tests that use the code.
We don't use unit testing because
1) it kills agility. We're small, new & rewrite our software base every year. Complete code coverage slows us down by a factor of two and doesn't give that time back (so far) in reduced dev time.
2) we have a graphical application that's hard to unit test.
3) it's shooting bugs in a barrel & assuming that because you don't hit a bug there aren't any.
Does it matter? The guys who made Twitter are unit testing fanatics and it doesn't seem to have helped their situation.
Unit testing is not load testing.
I understand that. It was a serious question, though. How worthwhile are extensive unit tests if your end product is still broken?
That's a very good point. IMO the biggest problem with unit tests is that so many people mis-use them, believing that they are a replacement for QA, integration testing, and load testing. You need people to look at your user interfaces to validate their usability, for one thing.
When I was working with Rails, I found the unit test framework to be nearly useless for my application since it relied on a SOAP service (and at least back then, you couldn't hit the service with a unit test), but the Rails integration tests were great.
So I ended up writing a fairly extensive suite of integration tests, but nearly no unit tests at all.
When I was working with Rails, I found the unit test framework to be nearly useless for my application since it relied on a SOAP service (and at least back then, you couldn't hit the service with a unit test), but the Rails integration tests were great.
So I ended up writing a fairly extensive suite of integration tests, but nearly no unit tests at all.
Yup, unit tests are a programmer aid, not a replacement for QA. If the unit tests aren't helping you code faster (by preventing debugging), they're not worth it.
It's probably close to impossible to prevent people from using unit tests as a substitute for QA, though. The concepts are similar enough that people are bound to confuse them, or at least take the confidence they get from passing unit tests as "good enough" to substitute for QA.
It's probably close to impossible to prevent people from using unit tests as a substitute for QA, though. The concepts are similar enough that people are bound to confuse them, or at least take the confidence they get from passing unit tests as "good enough" to substitute for QA.
The question is: How much worse would it be if they didn't have unit tests?
Let's hope.
In my experience with unit testing:
1. It doesn't help much if you are working in client side enginering. No matter how clever your unit test is, it can't help on detected ui rendering errors, animations, or multithreading programming.
2. Java + unit test, means you have to throw away all encapsulation, and good OO design to apease your unit tests, exposing your program to your coworkers to create havock by calling methods that shouldn't be visible to them.
3.Unit tests can give you a false sense of security. Oh, look, it passed the unit tests, my code must be good. Well, most of the time unit tests != real world scenarios.
4. TDD (test driven design) of software, can be slow. I would recommend it to all my competitors. In a startup, I'd rather release something fast, that might be still little buggy, than something that works well, but it took forever to get there. Plus TDD is constraining, as you are programming around your tests, and not around what the user will see. I found often, that my final design on a functionality, is different from how I thought it would be at the beginning. When working in large systems, you don't have the luxury to know how your final design will be.
5. Unit testing stiffles creativity in design, goes against "Exploratory programming". If you already know the answer on what you are building when you start, then you probably are not exploring around.
6. Unit testing != fun. Honestly, I'd rather be implementing features, than testing them.
Where unit testing is good:
1. When building an api that will be used by other people amd that is more static, less fluid. As oposed to UI enginering, something like geoCoding api's, have simple and clear results. That is very easy to unit test, as the value returned by the api is within a determined range, and the unit tests can be few, simple, and clear.
2. Scripting languages. Unit testing helps on detecting errors, where in a static language they would have been caught earlier. Plus it is much more easier to unit test on something like Python, then something like Java, or god forbid C/C++.
3. Coorporate enviroment. If you are waiting for something form a different group/department, you can build some unit tests to test if this group's code is meeting minimum standards.
Overrall Unit testing has its place, but be careful not to rely on it too much. It can't be substitue to real testing.