Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Channels ▼
RSS

Proof Positive


SD's Agile Modeling Newsletter: April 2005

SD's Agile Modeling Newsletter
April 2005
By Scott W. Ambler

In This Issue:

  • Proof Positive
  • Hot Links
    >> PROOF POSITIVE

    Solid research supporting the value of agility is beginning to come in, but the tough task of convincing the Doubting Thomases to "cross the chasm" may take further effort.

    It can be difficult to convince senior management that agile approaches are effective: Some people are honestly skeptical because they're afraid that agile software development is yet another fad, whereas others are happy with the status quo or fear change, demanding proof that agile techniques really work. Well, Thomas, doubt no more: Proof is starting to emerge.

    The "Technology Adoption Curve" presents critical insight into the proof question. First described by Geoffrey Moore in his groundbreaking book Crossing the Chasm (HarperBusiness, Revised Edition, 2002), the curve distinguishes among five types of technology adopters:

    1. Innovators who pursue new concepts aggressively.
    2. Early adopters who pursue new concepts very early in their lifecycle.
    3. The early majority who wait and see before buying into a new concept.
    4. The late majority who are concerned about their ability to handle a new concept, should they adopt it.
    5. Laggards who simply don't want anything to do with new approaches.
    Innovators and early adopters are comfortable thinking about a new concept for a bit and then tailoring it for their environment. Agile techniques are being adopted by organizations that fit these two profiles, and now some organizations in the early majority are testing their agile wings in pilot projects. Moore's concept of a chasm refers to the fact that some technologies or techniques work very well with innovators and early adopters, but are unable to make it to later phases in the adoption lifecycle. The agile community is clearly finding ways to cross the chasm: Agile techniques are now being applied on large teams (100-plus people); a certification process for Scrum masters is now offered via www.scrumalliance.org; and the agile community is actively reaching out to executives, as I noted in "Agility for Executives" (The Agile Edge, Sept. 2003). The demand for proof is generally made by people working in organizations that fit the late majority or laggard profiles, and, as agility becomes mainstream, we're going to hear that request more and more.

    Facts from the Past

    Agility's been around long enough now that a significant amount of proof is emerging. Craig Larman, in his new book Agile and Iterative Development: A Manager's Guide (Addison-Wesley, 2003), summarizes a vast array of writings pertaining to both iterative and incremental (I&I) development, two of agility's most crucial tenets, noting the positive I&I experiences of software thought leaders (including Harlan Mills, Barry Boehm, Tom Gilb, Tom DeMarco, Ed Yourdon, Fred Brooks and James Martin). More importantly, he discusses extensive studies that examine the success factors of software development. For example, he quotes a 2003 study conducted by Allen MacCormack and colleagues, to be published in IEEE Software, that looked at a collection of project teams of a median size of nine developers and 14 months' duration. Seventy-five percent of the project teams took an iterative and incremental approach, and 25 percent used the waterfall method. The study found that releasing an iteration's result earlier in the lifecycle seems to contribute to a lower defect rate and higher productivity, and also revealed a weak relationship between the completeness of a detailed design specification and a lower defect rate. Larman also cites a 2003 Australian

    study of agile methods, in which 88 percent of organizations found improved productivity, 84 percent experienced improved quality, 46 percent had no change to the cost of development, and 49 percent lowered costs. He also cites evidence that serial approaches to development, larger projects and longer release cycles lead to a greater incidence of project failure. A 2001 British study of 1,027 projects, for example, revealed that scope management related to waterfall practices, including detailed design up-front, was the single largest factor contributing to failure, cited by 82 percent of project teams.

    Barry Boehm and Richard Turner also explore the viability of agile approaches in Balancing Agility and Discipline: A Guide for the Perplexed (Addison-Wesley, 2003). They compare agile approaches with traditional approaches, which they call plan-driven approaches (a term I'm not fond of because planning does occur on agile projects), providing a risk-based approach to determine when to

    use each. Boehm and Turner's book reflects a reasonable, open-minded look at agile techniques by the industry's "old guard." "Agile methods," they write, "handle changeability and invisibility by building a shared vision of the project's objectives and strategies into each team member's shared store of tacit knowledge."

    Not TMI Yet

    Despite these encouraging offerings, the existing research isn't completely convincing. First, rather than specifically addressing agile approaches, much of it compares traditional waterfall approaches with ad hoc development. Second, although the two books mentioned effectively summarize the existing research, much about agile techniques remains to be examined. Agility's foundation is clearly solid, but we still must determine whether the agile methodologists have built something that will stand the test of time.

    Fortunately, many researchers are busy exploring agility's effectiveness. At this summer's XP/Agile Conference in New Orleans, a research workshop was organized by Grigori Melnik and Adam Geras of the University of Calgary and Laurie Williams of North Carolina State University to analyze the current state of ongoing empirical research into agile methods.

    In the workshop, participants reviewed and discussed three research projects, all of which are described in detail on the workshop website. Bill Krebs of North Carolina State University and IBM described his findings exploring the adoption of Extreme Programming (XP) by a development team. The most interesting part of his research was the suite of simple metrics that he used to judge the effectiveness of agile software development teams, such as unit test classes per user story, pairing frequency and user stories per person-month.

    Brian Hanks of University of California, Santa Cruz, discussed his study on the effectiveness of pair programming, in which he compared two classes of students taking the same course in different terms. One class was taught how to pair program and then asked to do so; the other wasn't. Although it's always hard to extrapolate student-based research to the commercial world, the results indicated that the pair-programming students learned the material better, supporting the claim that pair programming spreads knowledge across a team more effectively.

    Hakan Erdogmus of the National Research Council Canada described the work that he did with Maurizio Morisio, Marco Torchiano and Andrea Capiluppi of Politecnico di Torino in Italy comparing test-first and "test-last" development. It found that the test-first group, once again students, was 70 percent more productive than the group following a traditional approach. The test-first group also wrote 60 percent more tests than the test-last group, implying that the test-first mindset instilled greater discipline and thus led to greater productivity. Interestingly, on the final exam, the students reverted to a test-last approach, indicating that the culture shift required to adopt agile approaches is longer than a single school term.

    The anecdotal evidence certainly supports agile techniques, as does the foundational evidence gathered since the 1950s. W. H. Dana's 1993 study of NASA's X-15 hypersonic fighter project cited incremental delivery as a primary success factor ("The X-15 Lessons Learned," NASA Drysden Research Facility). It will take several years, perhaps even a decade, until we have incontrovertible proof that agile software development techniques work in practice. The research community clearly has its work cut out for it, and I hope that the IT's old guard allows and encourages these new lines of research. A handful of universities are leading the way; more will follow.

    —Scott W. Ambler

    This article was originally published in the November 2003 issue of Software Development.


    >>HOT LINKS

    For more information about determining when a model is agile, visit http://www.agilemodeling.com/essays/whenIsAModelAgile.htm

    For more information about determining when a document is agile, visit http://www.agilemodeling.com/essays/agileDocumentation.htm#WhenIsADocumentAgile

    To learn more about agile documentation, visit http://www.agilemodeling.com/essays/agileDocumentation.htm

    For overviews of a wide range of models, including all 13 UML 2 diagrams, visit the Agile Models Distilled page at http://www.agilemodeling.com/artifacts

    For tips on software modeling on whiteboards, read the essay posted at http://www.agilemodeling.com/essays/whiteBoardModeling.htm

    The Agile Alliance homepage is the best starting point for anyone interested in learning more about agile software development. http://www.agilealliance.org

    The principles of Agile Modeling v2 are described at http://www.agilemodeling.com/principles.htm

    The practices of Agile Modeling v2 are described at http://www.agilemodeling.com/practices.htm

    Check out the Agile Modeling mailing list at http://www.agilemodeling.com/feedback.htm

    Get agile modeling training resources at http://www.agilemodeling.com/training.htm


  • Related Reading


    More Insights






    Currently we allow the following HTML tags in comments:

    Single tags

    These tags can be used alone and don't need an ending tag.

    <br> Defines a single line break

    <hr> Defines a horizontal line

    Matching tags

    These require an ending tag - e.g. <i>italic text</i>

    <a> Defines an anchor

    <b> Defines bold text

    <big> Defines big text

    <blockquote> Defines a long quotation

    <caption> Defines a table caption

    <cite> Defines a citation

    <code> Defines computer code text

    <em> Defines emphasized text

    <fieldset> Defines a border around elements in a form

    <h1> This is heading 1

    <h2> This is heading 2

    <h3> This is heading 3

    <h4> This is heading 4

    <h5> This is heading 5

    <h6> This is heading 6

    <i> Defines italic text

    <p> Defines a paragraph

    <pre> Defines preformatted text

    <q> Defines a short quotation

    <samp> Defines sample computer code text

    <small> Defines small text

    <span> Defines a section in a document

    <s> Defines strikethrough text

    <strike> Defines strikethrough text

    <strong> Defines strong text

    <sub> Defines subscripted text

    <sup> Defines superscripted text

    <u> Defines underlined text

    Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

     
    Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.