Things I learnt by not writing tests(rfw.posterous.com)
rfw.posterous.com
Things I learnt by not writing tests
http://rfw.posterous.com/things-i-learnt-by-not-writing-tests
8 comments
I'll give my usual critique of religious TDD:
Most of the features I develop aren't good ideas in the end. Writing a test for a feature that you are going to dump after testing it with actual users is a waste of time.
Once a feature has proven its worth, I write tests to protect its value.
Most of the features I develop aren't good ideas in the end. Writing a test for a feature that you are going to dump after testing it with actual users is a waste of time.
Once a feature has proven its worth, I write tests to protect its value.
the problem with your argument is the hidden assumption that writing tests wastes time, when in practice it usually saves it. Long term benefit of tests is rarely denied, but as many of the TDD people can tell it usually saves time even in short term. when you develop a feature you are testing it, manually, by refreshing your browser and sometimes performing non-trivial sequences with it. Writing tests to do just that can save you time and leaves you with a better code.
[deleted]
I completely agree about the power of test-driven development, but I think the take-away hits home the most:
"...shaking things up, doing things a different way for awhile..., is never a bad idea."
I recently spent some time forcing a friend of mine to learn some basic Scheme (which I learned in school, and have always loved), to try his hand at programming. This led to basically reading though my Scheme book again and hacking through some examples. Going back to work - developing in Java - I found myself writing much more concise code and spending enough time thinking about the problem to find a solution that felt elegant.
Edit: I'd add to the article's point by saying: don't stay in the same environment for too long, or you'll get stale. Switch things up from time to time, even if that means switching to an environment that is inferior in some ways - you'll get something out of it.
"...shaking things up, doing things a different way for awhile..., is never a bad idea."
I recently spent some time forcing a friend of mine to learn some basic Scheme (which I learned in school, and have always loved), to try his hand at programming. This led to basically reading though my Scheme book again and hacking through some examples. Going back to work - developing in Java - I found myself writing much more concise code and spending enough time thinking about the problem to find a solution that felt elegant.
Edit: I'd add to the article's point by saying: don't stay in the same environment for too long, or you'll get stale. Switch things up from time to time, even if that means switching to an environment that is inferior in some ways - you'll get something out of it.
"Secondly, I've realised that when I don't write tests I code too fast." Excellent inadvertent criticism of TDD. Thanks for the post.
I'm not religiously in favor of "tests first, always" (who actually is?), but I find that when you emphasize testing, you spend less time debugging, more time coding, and the end result is generally better engineered.
After over a decade of writing software, I'm willing to accept this up-front cost in favor of long-term (and often short-term) gain. Sure, I can quickly hack out code when I'm not writing tests, but long term experience with both methodologies has demonstrated to me that it's not worth it.
After over a decade of writing software, I'm willing to accept this up-front cost in favor of long-term (and often short-term) gain. Sure, I can quickly hack out code when I'm not writing tests, but long term experience with both methodologies has demonstrated to me that it's not worth it.
> who actually is?
Like many other 'methodologies', it has uncritical advocacy. (http://blog.objectmentor.com/articles/2009/10/07/tdd-derange...) (HN discussion: http://news.ycombinator.com/item?id=866707)
I agree that automatic testing is nearly always a net gain, but using tests themselves as the primary driver for the design process suggests to me that the developer has little other design experience to draw upon.
Like many other 'methodologies', it has uncritical advocacy. (http://blog.objectmentor.com/articles/2009/10/07/tdd-derange...) (HN discussion: http://news.ycombinator.com/item?id=866707)
I agree that automatic testing is nearly always a net gain, but using tests themselves as the primary driver for the design process suggests to me that the developer has little other design experience to draw upon.
Actually, the full quote:
Secondly, I've realised that when I don't write tests I code too fast. TDD acts as a constraint on myself, something which stops me from running ahead of myself and forces me to stick around a particular problem area for long enough to understand it better, and often learn something that otherwise I'd have missed.
I don't remember ever reading a blog claiming that would be an advantage of TDD before I started.
Extreme Programming was cognizant of this effect quite a while ago. (Actually all of the parts of XP were supposed to have beneficial side effects that reinforced the other parts.)
Secondly, I've realised that when I don't write tests I code too fast. TDD acts as a constraint on myself, something which stops me from running ahead of myself and forces me to stick around a particular problem area for long enough to understand it better, and often learn something that otherwise I'd have missed.
I don't remember ever reading a blog claiming that would be an advantage of TDD before I started.
Extreme Programming was cognizant of this effect quite a while ago. (Actually all of the parts of XP were supposed to have beneficial side effects that reinforced the other parts.)
Can you explain more fully, please?
Presumably he's saying that if you don't write tests you can code faster.
I agree with the original poster -- it's important to slow down and think about the problem. Then again, I'm a big fan of trying hard not to write code you already understand too well -- that means that you're repeating yourself :-)
I agree with the original poster -- it's important to slow down and think about the problem. Then again, I'm a big fan of trying hard not to write code you already understand too well -- that means that you're repeating yourself :-)
In some cases, couldn't the tests be the code? That's the declarative programming ideal, anyway - starting with a straightforward spec of the problem, running it, and then gradually fleshing out just the parts that need to be optimized. Sometimes, writing what I want and then writing step-by-step directions for how to do it feels like I'm repeating myself.
Prolog is far from perfect (its default search mechanism gets stuck in left-recursive clauses), but it's a start, and computers have gotten considerably faster since the 70s...
AFAIK, the best known success of declarative programming is writing a pattern matching specification (with ML, Prolog, Erlang, Haskell, or various Lisp macros) and compiling it to the necessary nested if / switch statements, rather than generating it by hand. I guess it overlaps with parser generators, and some other DSLs, too.
Prolog is far from perfect (its default search mechanism gets stuck in left-recursive clauses), but it's a start, and computers have gotten considerably faster since the 70s...
AFAIK, the best known success of declarative programming is writing a pattern matching specification (with ML, Prolog, Erlang, Haskell, or various Lisp macros) and compiling it to the necessary nested if / switch statements, rather than generating it by hand. I guess it overlaps with parser generators, and some other DSLs, too.
One thing that discouraged me "Test-driven development is difficult to use in situations where full functional tests are required to determine success or failure. Examples of these are user interfaces, programs that work with databases, and some that depend on specific network configurations. TDD encourages developers to put the minimum amount of code into such modules and to maximise the logic that is in testable library code, using fakes and mocks to represent the outside world."
A good portion of interconnected web apps are API calls, and database updates/queries. For the type of programming I'm learning/doing now (web programming in ruby on rails), the tests are never sufficient to test anything. I fire up a browser and get a user eye view of what's happening while looking at the log. Maybe that's old school thinking but it's working well for us as this point.
Would appreciate any tips to help us be more efficient of course (victusmedia.com)
A good portion of interconnected web apps are API calls, and database updates/queries. For the type of programming I'm learning/doing now (web programming in ruby on rails), the tests are never sufficient to test anything. I fire up a browser and get a user eye view of what's happening while looking at the log. Maybe that's old school thinking but it's working well for us as this point.
Would appreciate any tips to help us be more efficient of course (victusmedia.com)
I had the same issue with TDD in Rails, where my functional tests were so brittle that they took more work than they saved, and my unit tests were so simple that I wasn't getting anything useful out of it.
What I really wanted to know was "will my site work," and I found Cucumber+Webrat to be an efficient way of being sure of that. I TDD Cucumber scenarios now, and it makes me more confident without feeling like I'm spending all my time fighting with a testing framework.
What I really wanted to know was "will my site work," and I found Cucumber+Webrat to be an efficient way of being sure of that. I TDD Cucumber scenarios now, and it makes me more confident without feeling like I'm spending all my time fighting with a testing framework.
I use TDD on all my projects. I also regard myself as extremely pragmatic in my work. If I felt I would do better without designing test-first I would not be doing it (on some projects I have less-than-full coverage because it felt like the investment was not worth it). On to my point...
One thing that always frustrates me in these debates is all the hypothesizing people who are skeptical of TDD do about what it would mean if they were to use it. For example:
"For the type of programming I'm learning/doing now (web programming in ruby on rails), the tests are never sufficient to test anything."
RoR is specifically built to make test-driven web development easy. I'm not saying the poster is wrong not to use TDD on their project, just that statements like that should not be taken at face value. It is clear that at least for some people doing certain types of work TDD is a big help. That's an indisputable fact. Therefore, the only reasonable positions one might hold are the following:
1) I agree with these people. TDD is great and I use it all the time. 2) I don't want to use TDD. It just does not match the way I think about my work. I choose not to use it, but I do not claim to know why using it is a bad idea. 3) I tried TDD and mastered it. Anyone well-versed in TDD would confirm that I can do it "the right way". I found the following problems in my test-driven code: ... For those reasons I would recommend that people stay away.
I think any other position would be disingenuous or lazy. Also note that only opinion 3 is in any way interesting to read about. It kind of feels like unless you're making that claim there is no useful way to comment on the utility of TDD.
One thing that always frustrates me in these debates is all the hypothesizing people who are skeptical of TDD do about what it would mean if they were to use it. For example:
"For the type of programming I'm learning/doing now (web programming in ruby on rails), the tests are never sufficient to test anything."
RoR is specifically built to make test-driven web development easy. I'm not saying the poster is wrong not to use TDD on their project, just that statements like that should not be taken at face value. It is clear that at least for some people doing certain types of work TDD is a big help. That's an indisputable fact. Therefore, the only reasonable positions one might hold are the following:
1) I agree with these people. TDD is great and I use it all the time. 2) I don't want to use TDD. It just does not match the way I think about my work. I choose not to use it, but I do not claim to know why using it is a bad idea. 3) I tried TDD and mastered it. Anyone well-versed in TDD would confirm that I can do it "the right way". I found the following problems in my test-driven code: ... For those reasons I would recommend that people stay away.
I think any other position would be disingenuous or lazy. Also note that only opinion 3 is in any way interesting to read about. It kind of feels like unless you're making that claim there is no useful way to comment on the utility of TDD.
I find your logic interesting.
So let's say I have this new way of doing things. It's called "hit your head with a hammer". I'm crazy about it and wrote a bunch of books about it. Other people have tried hitting their head with a hammer and, although there are many detractors, there are also some big fans.
So what you're saying is: we cannot make a legitimate (or "interesting") observation about "hit yourself in the head with a hammer" unless we also have mastered the art of doing it. Only then would we have something worthwhile to add.
I'm not saying TDD is bad. What I'm saying is that it's perfectly fine to look at the phenomenon from the outside and make general observations about it. For instance, "TDD is very hard to impose on most teams" or "When implementing TDD for the first time, teams experience a significant drop in performance"
I'm sure there are lots of others.
Uncle Bob (and others) have made a mockery out of informed discussion in regards to TDD. Here's hoping we can keep a reasoned and civilized discussion going in the community.
So let's say I have this new way of doing things. It's called "hit your head with a hammer". I'm crazy about it and wrote a bunch of books about it. Other people have tried hitting their head with a hammer and, although there are many detractors, there are also some big fans.
So what you're saying is: we cannot make a legitimate (or "interesting") observation about "hit yourself in the head with a hammer" unless we also have mastered the art of doing it. Only then would we have something worthwhile to add.
I'm not saying TDD is bad. What I'm saying is that it's perfectly fine to look at the phenomenon from the outside and make general observations about it. For instance, "TDD is very hard to impose on most teams" or "When implementing TDD for the first time, teams experience a significant drop in performance"
I'm sure there are lots of others.
Uncle Bob (and others) have made a mockery out of informed discussion in regards to TDD. Here's hoping we can keep a reasoned and civilized discussion going in the community.
I agree. I would say the observations you give as examples are backed by evidence and thus interesting to read (and, in fact, legitimate). I stated my case badly, but I would include such statements in the third category. I meant what I said to be a call for civilized discussion about the topic which echoes your sentiment. I also wanted to point out that such discussions have an annoyingly low signal-to-noise ratio.
[Edit] I also was not trying to suggest a general strategy for evaluating programming practices. I made the assertion it was clear TDD has value at least in certain contexts. In the case of TDD I think it is evident its benefits are subtle enough to approach the issue more carefully.
[Edit] I also was not trying to suggest a general strategy for evaluating programming practices. I made the assertion it was clear TDD has value at least in certain contexts. In the case of TDD I think it is evident its benefits are subtle enough to approach the issue more carefully.
While I have tried it, my experience with TDD is quite limited.
With that said, you seem to be missing an option:
4. I have evaluated TDD, and I see its value in many cases. But I also see cases where it is not appropriate or at least suboptimal. I therefore consider on a project by project basis whether I should use it.
My experience in TDD is nowhere near enough to be comfortable making statement 4 myself, but I have heard it made and I think even most people that disagree with it would see it as a reasonable position to hold.
With that said, you seem to be missing an option:
4. I have evaluated TDD, and I see its value in many cases. But I also see cases where it is not appropriate or at least suboptimal. I therefore consider on a project by project basis whether I should use it.
My experience in TDD is nowhere near enough to be comfortable making statement 4 myself, but I have heard it made and I think even most people that disagree with it would see it as a reasonable position to hold.
Why not test rails apps?? That's the bit I don't get... for starters get a hold of sahi http://sahi.co.in/w/ (other web test tools also exist...). Voila simple tdd in the browser environment.
In defence of the article, tdd is something i do in my head but rarely to the nth degree in practice. I'm more of a fdd kind of person, but I think that there are a lot of parallels with the two approaches.
In defence of the article, tdd is something i do in my head but rarely to the nth degree in practice. I'm more of a fdd kind of person, but I think that there are a lot of parallels with the two approaches.
In a similar vain to what the author is talking about by writing tests you probably end up creating a better api, especially if you write the tests first.
Well, sure. Designing an API, informed by feedback from writing some tests as you go, and dogmatically insisting that it's best to always write tests first and let that be the primary driver of the design process are two different things, however. (Not implying that you're saying the latter.)
I don't think a lot of people would argue that writing tests can be a good way to get feedback about a design. Some particularly vocal TDD advocates seem to be making it into an all-or-nothing issue, though.
I don't think a lot of people would argue that writing tests can be a good way to get feedback about a design. Some particularly vocal TDD advocates seem to be making it into an all-or-nothing issue, though.
That's called design by contract:
http://en.wikipedia.org/wiki/Design_by_contract
I learned exactly the same lessons from writing code in a functional style. The experience changed my style for the better, and that carries over even when I'm using a language that doesn't lend itself very well to functional programming.