Number of Tests Written
We wrote 45 individual unit tests for the features developed with TDD. The goal was not to have 100-percent test coverage but to create a solid design that would lend itself to easy updating. By not focusing solely on the number of test cases written, or requiring that tests be run on every official build, we made gains in technology areas that traditionally would not have been unit tested.
For example, the SSO (single sign-on) feature boils down to Kerberos integration. Kerberos is a secure authentication and authorization mechanism that is somewhat rigid in its client computer configuration requirements. Those requirements make it difficult to set up a process where Kerberos can be tested from any computer (for instance, usernames and passwords of Kerberos users would need to be stored where the unit tests could read them, creating a major security risk).
However, the design benefits that TDD brought to the other features were too good to pass up and we wrote the Kerberos feature in the same manner. We commented out its unit tests out once the code was checked into source control. The tests can be run when this area of functionality changes but they are not run as part of the standard build process.
It could be argued that all the Kerberos server functionality could be simulated using Fake or Mock objects but in the end that was determined to be too much extra work for a minimal amount of gain.
Empirical Quality of the Code
Code quality is always a touchy subject in the development world, with many different opinions on what is good and what is bad. According to the developer who wrote it, the code developed using TDD seems to be better than the code written in the traditional manner for the following reasons:
- The TDD code was refactored several times during development. Methods were created, then changed, and then changed again with each revision, resulting in code that was tighter or clearer. The developer felt like the code went through an extensive proofreading cycle similar to what is done for newspapers, magazines, or the like.
- There was a freedom of experimentation that led to better design decisions. Once the first working version of a class method was completed and properly tested, the developer could then try new design ideas. The unit tests quickly confirmed or denied that the new code worked, providing a digital safety net.
- On a number of occasions, the developer found that the TDD method forced him to slow down and think more carefully about what he was trying to accomplish. Rather than just banging out code to get the job done, he focused on writing high quality code.
Conclusion
It is clear the TDD method resulted in fewer defects than the traditional method. The time difference between the two methods is difficult to measure because of the difference in features. However, an educated estimate has TDD development taking 15-20 percent longer than its traditional counterpart. It is certainly possible that this number will go down after the developer has more experience with TDD. However, it is worth mentioning it as a cost required for anyone trying TDD for the first time.
This project did not measure whether or not TDD leads to long term gains in quality. All indications suggest that TDD might be a compelling tool and it certainly warrants further investigation within your own development efforts.
