Comments on: Misconceptions with Test Driven Development http://agilepainrelief.com/notesfromatooluser/2008/11/misconceptions-with-test-driven-development.html Best practices for your goals Fri, 03 Sep 2010 00:32:47 +0000 http://wordpress.org/?v=2.9.1 hourly 1 By: Adam Sroka http://agilepainrelief.com/notesfromatooluser/2008/11/misconceptions-with-test-driven-development.html/comment-page-1#comment-132 Adam Sroka Tue, 30 Nov 1999 00:00:00 +0000 http://agilepainrelief.com/2008/11/misconceptions-with-test-driven-development/#comment-132 I have made contributions to both open and closed source projects where all of my code was test-driven even if some (or all) other code was not. I find that TDD fits my brain. I also find that the code that *I* produce is better/simpler/more correct with TDD (And even better/simpler/more correct with close collaboration with at least one pair programmer.) However, it is hard to quantify how/if this improves the overall quality and design in an environment where these practices aren't strictly enforced. It is hard enough to know what percentage of the code benefits both directly and indirectly from the introduction of TDD let alone what percentage of suck-ness results directly and indirectly from where it is missing. I have made contributions to both open and closed source projects where all of my code was test-driven even if some (or all) other code was not.

I find that TDD fits my brain. I also find that the code that *I* produce is better/simpler/more correct with TDD (And even better/simpler/more correct with close collaboration with at least one pair programmer.)

However, it is hard to quantify how/if this improves the overall quality and design in an environment where these practices aren’t strictly enforced. It is hard enough to know what percentage of the code benefits both directly and indirectly from the introduction of TDD let alone what percentage of suck-ness results directly and indirectly from where it is missing.

]]> By: Kevin Rutherford http://agilepainrelief.com/notesfromatooluser/2008/11/misconceptions-with-test-driven-development.html/comment-page-1#comment-133 Kevin Rutherford Tue, 30 Nov 1999 00:00:00 +0000 http://agilepainrelief.com/2008/11/misconceptions-with-test-driven-development/#comment-133 reek, the code smells detection tool for ruby, is being developed entirely using TDD. See the source at http://github.com/kevinrutherford/reek and the docs at http://reek.rubyforge.org/rdoc reek, the code smells detection tool for ruby, is being developed entirely using TDD. See the source at http://github.com/kevinrutherford/reek and the docs at http://reek.rubyforge.org/rdoc

]]> By: Dave Nicolette http://agilepainrelief.com/notesfromatooluser/2008/11/misconceptions-with-test-driven-development.html/comment-page-1#comment-134 Dave Nicolette Tue, 30 Nov 1999 00:00:00 +0000 http://agilepainrelief.com/2008/11/misconceptions-with-test-driven-development/#comment-134 It's interesting that many of those who question the effectiveness of TDD never seem to question the effectiveness of traditional methods, which are responsible for the dismal performance of the IT industry over the past few decades as documented in studies by Gartner, Forrester, Standish, and others. They demand newer methods like TDD have to prove themselves as compared with...well, with <i>what</i>, exactly? What's their baseline for comparison? Long lead times, high defect rates, untenable design debt, cost overruns? Rather than asking whether TDD "really works," they ought to ask whether any other approach has really worked. It’s interesting that many of those who question the effectiveness of TDD never seem to question the effectiveness of traditional methods, which are responsible for the dismal performance of the IT industry over the past few decades as documented in studies by Gartner, Forrester, Standish, and others. They demand newer methods like TDD have to prove themselves as compared with…well, with what, exactly? What’s their baseline for comparison? Long lead times, high defect rates, untenable design debt, cost overruns? Rather than asking whether TDD “really works,” they ought to ask whether any other approach has really worked.

]]> By: Kane Mar http://agilepainrelief.com/notesfromatooluser/2008/11/misconceptions-with-test-driven-development.html/comment-page-1#comment-135 Kane Mar Tue, 30 Nov 1999 00:00:00 +0000 http://agilepainrelief.com/2008/11/misconceptions-with-test-driven-development/#comment-135 Surprisingly, there is little clear evidence that TDD is beneficial. After a question was raised on a forum that I regularly read, I spent several days reading the current literature on TDD ... case studies, reports, recommendations etc. The vast majority of studies and reports simply don't reach any firm conclusions. And, for every article that promoted certain properties of TDD there are an equal number of articles demoting those same properties. All this came as a surprise to me because I had assumed that the benefits of TDD were a forgone conclusion ... based on personal experience and practice of TDD. What is very clear is that there is a correlation between the number of tests and clarity (simplicity) of the code. Quoting from your article: "measuring a over 20 projects: if you have a large number of unit tests your code will be an order of magnitude less complex." I would like to point out that one doesn't need to practice TDD, to have a lot of unit tests. There are other ways to achieve the same end. I absolutely agree with the sentiment of your post and Dave's comments above, especially the following quote: "It's interesting that many of those who question the effectiveness of TDD never seem to question the effectiveness of traditional methods ..." I would also like to point out that the benefits of TDD are not clear nor are they immediately apparent. Best regards, Kane Mar <a href="http://Scrumology.com" rel="nofollow">http://Scrumology.com</a> Surprisingly, there is little clear evidence that TDD is beneficial. After a question was raised on a forum that I regularly read, I spent several days reading the current literature on TDD … case studies, reports, recommendations etc. The vast majority of studies and reports simply don’t reach any firm conclusions. And, for every article that promoted certain properties of TDD there are an equal number of articles demoting those same properties.

All this came as a surprise to me because I had assumed that the benefits of TDD were a forgone conclusion … based on personal experience and practice of TDD.

What is very clear is that there is a correlation between the number of tests and clarity (simplicity) of the code. Quoting from your article: “measuring a over 20 projects: if you have a large number of unit tests your code will be an order of magnitude less complex.” I would like to point out that one doesn’t need to practice TDD, to have a lot of unit tests. There are other ways to achieve the same end.

I absolutely agree with the sentiment of your post and Dave’s comments above, especially the following quote: “It’s interesting that many of those who question the effectiveness of TDD never seem to question the effectiveness of traditional methods …” I would also like to point out that the benefits of TDD are not clear nor are they immediately apparent.

Best regards,
Kane Mar
http://Scrumology.com

]]> By: Mark Levison http://agilepainrelief.com/notesfromatooluser/2008/11/misconceptions-with-test-driven-development.html/comment-page-1#comment-136 Mark Levison Tue, 30 Nov 1999 00:00:00 +0000 http://agilepainrelief.com/2008/11/misconceptions-with-test-driven-development/#comment-136 Have you looked into the most recent study from MS - that shows a reduction in the defect rate? I don't remember the details right this second but I think George Dinwiddie links to it. Cheers Mark Have you looked into the most recent study from MS – that shows a reduction in the defect rate? I don’t remember the details right this second but I think George Dinwiddie links to it.

Cheers
Mark

]]> By: Kane Mar http://agilepainrelief.com/notesfromatooluser/2008/11/misconceptions-with-test-driven-development.html/comment-page-1#comment-137 Kane Mar Tue, 30 Nov 1999 00:00:00 +0000 http://agilepainrelief.com/2008/11/misconceptions-with-test-driven-development/#comment-137 Yes, absolutely. And as part of their conclusion they state: "Drawing general conclusions from empirical studies in software engineering is difficult because any process depends to a large degree on a potentially large number of relevant context variables. For this reason, we cannot assume a priori that the results of a study generalize beyond the specific environment in which it was conducted." I'm not disputing that TDD is beneficial. I *am* saying that the benefits are neither clear, nor obvious ... at least at this point in time. In several years time (with a body of evidence) this discussion might appear dated and quaint. As of today (Jan, 2010) I think it's fair to say the the jury is still out on TDD. Best regards, Kane Mar <a href="http://Scrumology.com" rel="nofollow">http://Scrumology.com</a> Yes, absolutely. And as part of their conclusion they state: “Drawing general conclusions from empirical studies in software engineering is difficult because any process depends to a large degree on a potentially large number of relevant context variables. For this reason, we cannot assume a priori that the results of a study generalize beyond the specific environment in which it was conducted.”

I’m not disputing that TDD is beneficial. I *am* saying that the benefits are neither clear, nor obvious … at least at this point in time. In several years time (with a body of evidence) this discussion might appear dated and quaint. As of today (Jan, 2010) I think it’s fair to say the the jury is still out on TDD.

Best regards,
Kane Mar
http://Scrumology.com

]]>