Extreme Testing

Test-driven development (TDD), also known as test-first programming or test-first development, is an iterative approach that hinges on identifying and writing a single test prior to writing just enough functional code to break that test. Although it isn’t obvious from the name, in Agile Software Development (Prentice Hall, 2002), Robert Martin points out that TDD’s goal is actually one of specification, not validation. In other words, it’s one way to think through your design before you write your functional code.

TDD employs five basic steps. First, you quickly add a test (just enough code to fail). You can’t write new functional code—even a single line—unless you’ve run a test that fails without it. The second step involves running your tests, either all or some, to see the new test fail. Third, you make a little change to your functional code—just barely enough to enable your code to pass the tests. Next, you run the tests and, if you’ve lined up all your ducks in a row, see them all succeed—if something waddles out of place, you need to repeat step three. In the optional fifth step, you refactor your code to remove any duplication.

Kent Beck, who popularized TDD in Extreme Programming (XP), defines two simple rules. First, you should write new business code only when an automated test has failed. Second, you should eliminate any duplication that you find. Beck explains how these two simple rules generate complex individual and group behavior:

• You design organically, with the running code providing feedback between decisions.
• You write your own tests because you can’t wait 20 times a day for someone else to write them for you.
• Your development environment must provide rapid response to small changes (for this, you need a fast compiler and regression test suite).
• Your designs must consist of highly cohesive, loosely coupled components (your design is highly normalized) to make testing easier—which makes evolution and maintenance of your system easier, too.

TDD’s most significant benefit is its efficiency in the real world. In his new book, Test-Driven Development (Addison-Wesley 2003), Beck reports working on a Smalltalk system with a completely test-driven approach that took four years and 40 person-years of effort, resulting in 250,000 lines of functional code and 250,000 lines of test code. 4,000 tests ran in less than 20 minutes, with the full suite being run several times a day. Although there are larger systems out there, this project makes it clear that TDD can work for nontrivial systems.

A Replacement for Traditional Testing?
Although TDD is a programming technique, you also gain 100 percent unit-test coverage of your code if it’s done correctly—an important by-product. As we all know, there’s far more to testing than unit tests, so you’ll still need other testing techniques such as functional testing, user acceptance testing, system integration testing and so on.

With TDD, as with traditional testing, a successful test finds one or more defects, and when a test fails, you’ve made progress because you now know that you need to resolve the problem. More importantly, when the test no longer fails, you have a clear measure of success. With TDD, you can determine if your system actually works and meets the requirements defined for it, thus enabling you to proceed with confidence.

A Replacement for Modeling?
How does TDD stack up to model-driven development (MDD), or, more specifically, agile model-driven development (AMDD)? See “Side by Side: Comparing TDD and AMDD” for some answers. Both TDD and AMDD are based on the necessity of thinking through your design before you code. With TDD, you do so by writing tests, whereas with AMDD, you think ahead by creating diagrams or other types of models such as Class Responsibility Collaborator (CRC) cards.

Which approach is better? It depends on your team’s cognitive preferences. Visual, spatial thinkers are best served by graphic means. Verbal thinkers learn better from text than from drawings. Keep your team members in mind when you determine your choice of development approach.

Can you combine the two approaches? Absolutely. I’ve enhanced TDD with modeling to great success, creating models with project stakeholders to help explore their requirements and then moving on to a test-driven approach for implementation. I prefer to use AMDD approaches for 10 or 15 minutes of “up-front work” and then TDD for finely grained development work (often for hours at a time).

Tests as Documentation
Like it or not, most programmers don’t read the written documentation for a system; instead, they prefer to dig right into the code. This is OK if you’re willing to act accordingly. When trying to understand a class or operation, most programmers will look for sample code that already invokes it. Well-written unit tests do exactly this.

Similarly, many XPers argue that acceptance tests form an important part of requirements definition. This makes a lot of sense. Your acceptance tests define exactly what your stakeholders expect of your system; therefore, they truly do determine a good portion of your requirements.

Testing Tools
TDD is based on the underlying assumption that you have a unit-testing framework available. Agile software developers often use the xUnit family of open-source tools, such as JUnit or VBUnit, although commercial tools such as Parasoft’s JTest and SilverMark’s Test Mentor are also viable options. Without good tools, TDD is virtually impossible.

Step by Step
TDD helps you to take small steps when writing software, a practice that I’ve promoted for years—baby steps are far more productive than headlong leaps. For example, assume that you add some new functional code, compile and test it. It’s likely that your tests will be broken by defects that exist in the new code. To find and fix those defects, it’s much easier to hunt through two instead of 2,000 new lines of code. The faster your compiler and regression test suite, the more attractive it is to proceed in smaller and smaller steps. I generally prefer to add a few new lines of functional code, typically fewer than 10, before I recompile and rerun my tests.

I think Bob Martin says it best: “The act of writing a unit test is more an act of design than of verification. It’s also more an act of documentation than of verification. The act of writing a unit test closes a remarkable number of feedback loops, the least of which is the one pertaining to verification of function.” Ron Jeffries provides a complementary view, saying that the goal of TDD is simply to write clean code that works.

Scott W. Ambler is a senior consultant with Ronin International Inc. His latest book is The Elements of UML Style (Cambridge University Press, 2002).