Book Review: The Practical Guide To Defect Prevention

Over the holidays I received a review copy of The Practical Guide To Defect Prevention, by Ross Smith, Marc McDonald, and Robert Musson. I have worked with Ross in the past, and I knew he and his team have been doing some interesting work in this area, so I was eager to learn more. I was also looking forward to hearing all sorts of fun stories describing how they have applied these ideas within Microsoft.

Alas, while details were copious stories were sorely lacking. This is my biggest (only, really) complaint about this book. The many ideas it contains sound great, yet without real-life experiences backing them up I am left wondering how many of them have actually been applied, whether they worked or not, and what the authors would do differently next time. Maybe there's an action-packed sequel on the way...

That said, I found this Practical Guide to be chockablock full of interesting techniques for stopping defects before they start. Ideas I found especially interesting include:

• Applying game theory to the problem of incenting people to do tasks they don't really want to do, such as installing a new build of your product every day


• Combining source control churn and function dependency data with various other bits of information to provide a guesstimate as to the riskiness of a specific code change (please tell me this will be part of Visual Studio Team System soon!)


• Using simulation and modeling to optimize your software development process


• Failure Mode, Effects and Criticality Analysis [FMEA] and Fault Tree Analysis [FTA], which appear to me to be more sophisticated (and exotic) forms of the risk-based testing many testers do today

Also covered is the people side of defect prevention. Suggestions for creating a culture of quality, moving quality upstream, and improving communication abound. However, I felt it to be more a series of "you might want to learn about this" pointers than in-depth explanations. Additionally, as with the rest of the book, while some of the ideas (e.g., "Product Information Is Executable") sound wonderful I was unable to determine whether they have actually been implemented or are merely the authors' idea of the way things ought to be. (If the former, please tell me where to buy this magic!)

My favorite part of the book was the three short paragraphs which describe how the Windows Vista team had a small team of elite testers who manually ran through a set of scenario-based test cases for every build. "In the end, the cost of the seemingly 'inefficient' manual testers helped improve the efficiency of the whole system. The return on the small investment was huge. There is a place for manual testing." I am going to be plopping this bit of this book on multiple desks!

I also liked the idea (taken from Dave Olson's Exploiting Chaos) of having one team do prototyping and initial development work and no production work, while another team uses the prototype as a guide in writing the production version. Developers who are skilled at prototyping are often not skilled at polishing, and vice versa; this model leverages the strengths of each. (Although, again, I am left wondering whether this has actually been implemented by anybody, and if so how well it worked.)

Looking at the book as a whole, I think The Practical Guide To Defect Prevention is worth reading if only to get your brain pondering the possibilities. My advice is to flip through the book quickly, casting your eyes over, well, whatever catches your eye. Next, have a slow read through the table of contents, again taking a longer look at topics which stand out to you. Then put the book aside until that point in the future when you are faced with a thorny problem which might be solvable using some of the tools and techniques in this book. That's the time to pull out this Practical Guide and give the relevant portions a thorough going-over. (Their website too: defectprevention.org.)

Posted by The Braidy Tester at 07:30 AM  Permalink