Assurance & Agile Processes

Assurance that systems will perform as you expect does not conflict with agility—it merely challenges it.


June 01, 2006
URL:http://drdobbs.com/architecture-and-design/assurance-agile-processes/188700785

Cliff is founder of Assured By Design and author of High-Assurance Design (Addison-Wesley Professional, 2005). He can be contacted at [email protected]

Scott is a software process improvement consultant, mentor, and trainer with Ambysoft Inc. He can be contacted at www.ambysoft.com.


For organizations of all sizes, security failures can have devastating consequences. But security is only one aspect of overall application reliability. Indeed, it is often possible to equate the cost of a security failure with a given level of downtime. Thus, overall assurance of an application—that is, its ability to perform as required and resist or recover from failure of any kind—is really the key issue that organizations need to consider.

While the improved productivity achieved by agile software development practices is compelling, agile methods such as Extreme Programming (XP) do not explicitly address security and reliability. Why not? Because the assumption is that these issues will be addressed by the project's customers. Some have addressed the issues from the perspective of testing (www.testing.com/writings/ evaluating-test-suites-paper.pdf) and design (Agile Modeling: Effective Practices for Extreme Programming and the Unified Process, by Scott Ambler, Wiley, 2002) or security (e.g., Beznosov, Towards Agile Security Assurance, presented at www.nspw.org/2004). Still, there needs to be a cohesive agile framework that addresses assurance.

The Importance of Design for Assurance

To avoid constantly saying "architecture and design," we use the terms "architecture" and "design" interchangeably. After all, an architecture is merely a high-level design, and for purposes here the distinction is not important.

Newcomers to some agile methods think that design is passé, a myth dispelled in many sources (www.martinfowler.com/ieeeSoftware/ whoNeedsArchitect.pdf). In Agile Modeling: Effective Practices for Extreme Programming and the Unified Process, one of us (Scott) explains why long-term objectives need to be considered if they are important to the organization. For example, objectives such as maintainability may mean that certain kinds of documentation are required as deliverable artifacts. The key is to create "agile documents" (www.agilemodeling.com/essays/ agileDocumentation.htm).

Let's be clear about what a design is. A design is an enumeration or description of elements or attributes of an application's intended manifestation. Design may also address semantic elements, such as relationships between structural elements, sequence, cardinality, and in general, any aspects important for implementers to know to be able to produce an implementation that adheres to the design's intent.

The embodiment of a design is a set of artifacts. The artifacts can be code, unit tests, diagrams, or documented decisions about important technical issues. These are not mutually exclusive.

In 2004, Peter Neumann stated, "Good system and network architecture is perhaps the most fundamental aspect of any efforts to develop trustworthy systems..." (www.csl.sri.com/users/ neumann/chats4.html). The reason behind this belief is that a design embodies the concepts that ensure that the system has certain hard-to-test properties, such as being secure. The design addresses how these properties are achieved, via the components of the design and their required relationships.

A design should not only state what elements exist in an application, but it should also state why they exist. For example, if a particular software layer exists to protect an underlying resource, then the layer should not be eliminated in order to simplify the design. The layer exists to satisfy a security requirement, and the layer's design should state how that is achieved. To explain how, you must ask "why?"; otherwise, you are explaining how something happens that you do not even have an expressed justification for.

There is another reason that "why" is important: It expresses a conclusion about the adequacy of a feature. That conclusion is then open to challenge. Agile design is about evolving a solution over time, and about continually challenging each "why" and its associated conclusions.

Emergent Properties

In the foreword to High Assurance Design (by Cliff Berg; Addison-Wesley Professional, 2005), Neumann states that "Security and reliability are both emergent properties of an entire system..." A consequence of this is that many kinds of tests that must be performed to verify security and reliability are so-called "negative tests" that verify that something cannot be done, rather than verifying that something can be done. To do these kinds of testing, a complex testing configuration might be necessary that is best left in the hands of professional testers. As Brian Marick put it, "good programmers do functional testing, and testers should do everythingelse" (www.testing.com/writings/ classic/mistakes.html).

While considering a holistic system design, you must be careful to avoid falling into the pit of "big design up-front" (BDUF). You should be able to accomplish this if you follow agile principles regarding how the design is developed. Specifically, the criteria in Table 1 should be met for a design to be "agile."

1. Create the minimal design that is necessary at the time.
2. Maintain only those parts of the design that need to be maintained to satisfy the mission of the application—Update Only When It Hurts (see Agile Modeling).
3. Define success criteria—tests—that the design must meet from the beginning, in a manner analogous to test-driven development (TDD), and require the design to be "tested" with each build of the system.

Table 1: Agility criteria for design.

As an example of item 3 in Table 1, consider an application for which the design includes a layer to protect an underlying resource, and for which the design includes a rule that "no module may directly access the resource except for the protection layer." Any code that violates this rule puts the entire protection scheme at risk. Therefore, verification of compliance with this rule must be achieved if it is to be successful. Because this rule cannot be tested by using functional tests, another means must be used. The testing of this design rule—its verification—can be achieved through any or several means, including the methods in Table 2.

1. Manual inspection of the entire code that might be able to directly access the resource.
2. Automated inspection, using code scanning or parsing tools.
3. Runtime checking, using dynamic tools.
4. Sufficient awareness on the part of developers, so that they know not to violate the rule.

Table 2: Design compliance verification methods.

The choice should be made in consideration of the project's unique circumstances, including the level of maturity of the team, the size of the project, and the complexity of the overall code base.

Methodological Challenges

Some agile methodologies such as XP have raised questions among security and reliability experts. Their questions primarily focus around the concerns in Table 3.

1. Need for a focus on a comprehensive design.
2. Need for a focus on documentation of the design and its intent.
3. Need for a focus on nonfunctional testing.
4. Need for a separation of high-risk modules, in light of "collective code ownership."

Table 3: Assurance concerns often raised for agile methods.

For example, in their paper "Modeling and Implementing Software Architecture with Acme and ArchJava," Abi-Antoun et al. assert that the benefits of architecture are "contingent upon correct implementation [with respect to] program understanding, software evolution, checking architectural constraints, [and] analysis of quality attributes" (www.cs.cmu.edu/~aldrich/ papers/icse05-demo.pdf). This is a widely held view in the security and software assurance communities.

Items 1 and 2 in Table 3 have to do with defining, disseminating, and perpetuating architectural intent. "Intent" is the "why" that we previously discussed. Because TDD focuses on specification in the form of a unit test suite, the TDD-style of checking architectural intent would be to implement architectural rules as a test suite. A less formal approach is for the architects to be active members of the development team, thereby ensuring that design intent is adhered to because they're actually building it themselves (see The Practical Guide to Enterprise Architecture, by James McGovern et al., Prentice Hall PTR, 2003). Still, the assumptions and concepts—the "why"—should be written down and maintained.

Agile Modeling (AM) was specifically designed to be tailored into other agile methods to address the modeling and documentation issue. Similarly, TDD can be tailored to address assurance testing issues (www.testing.com/cgi-bin/blog/2003/08/ 21#agile-testing-project-1).

Agile Assurance

To address assurance from an agile perspective, the practices in Table 4 should be followed.

1. When a requirement cannot be effectively verified using execution tests, use the design (not the code) as the focus of evidence of correctness and completeness.
2. Ensure that requirements that are collected include assurance objectives.
3. Use test-driven development and other techniques to continually verify compliance of the implementation with ongoing design (the AM effort; see below) as well as with requirements.
4. Augment 3 with randomized testing to empirically assess actual assurance.

Table 4: Agile assurance practices.

Thus, for assurance-related requirements, our focus shifts to design rather than implementation, leaving implementation as a concurrent activity. A software implementation is merely a low-level design, and if a high-level design must remain authoritative for some aspects, then the low-level design is, by definition, subordinate to it for those aspects. We use the term "agile assurance" (AA) to refer to our approach.

To explain this approach, we discuss its implementation within the context of XP. Let's take the elements of our agile assurance approach, one by one.

Design as Evidence. Approach 1 in Table 4 states that the design is a focus on an AA effort. A design is not an objective in its own right; rather, a design is an instrument for verifying that assurance is achieved. Further, a design must be adequately documented, to the extent necessary to achieve assurance objectives, including proving that the code agrees with the design intent, and ensuring that future maintenance also complies with design intent. For this to be achievable, the design must be maintained in a concrete and durable form, such as unit tests, a document, code comments, rules specified via a design language, or any means appropriate for a project. Thus, because a high-assurance design must be sufficiently documented, in the rest of this article we use the term "design" as synonymous with "design specification."

Here, we propose an approach for capturing design (www.agilemodeling.com/ essays/amdd.htm). Agile Model Driven Development (AMDD) is the application of the principles and practices of AM on an agile project.

Identification of Stakeholders and Requirements Collection. In AMDD, stakeholders are defined as anyone who is a direct user, indirect user, manager of users, senior manager, operations staff member, support (help desk) staff member, developers working on other systems that interact with the one under development, or maintenance professionals potentially affected by the development/deployment of a software project (www.agilemodeling.com/essays/ activeStakeholderParticipation.htm).

Requirements such as security, reliability, failure recovery, disaster recovery, maintainability, and manageability cannot be treated as implicit. An agile design in a high-assurance environment requires that all assurance requirements be expressed through stakeholder stories. This requires that all of the application's stakeholders must be represented in collecting requirements. Those who have the most dynamic requirements—typically the end users—should be met with each iteration.

Continual Design Verification Augments TDD. Agile proponents know the value of test-driven design (TDD), in which a test is written and then enough production code is written to fulfill that test. Again, a primary purpose of a design should be to explain why certain features exist in the application. AMDD addresses TDD by differentiating between high-level design, which utilizes agile modeling, and detailed design, which uses TDD to define detailed behavior.

An agile design must be continually verified, in the same manner that the implementation must be continually tested. Verification must occur from two perspectives:

Empirical Testing. The very nature of assurance requirements, in that they tend to be negative requirements specifying that something cannot be done, means that it is difficult to create a high-coverage test suite. This means that some amount of empirical testing, with inputs that are randomly chosen, is necessary to measure the actual soundness of the application so that it can be compared with the designed soundness.

Agile Modeling

Development of the Agile Modeling (AM) methodology was led by one of us (Scott). Agile Modeling is a collection of values, principles, and practices for modeling software that can be applied on a software-development project in an effective and lightweight manner. AM provides a framework of practices for developing and maintaining an application design. AM is meant to be tailored into other, full-fledged methodologies such as XP, RUP, or the Agile Unified Process (www.ambysoft.com/services/ agileUP.html), letting you develop a software process that truly meets your needs.

Agile Model Driven Development (AMDD). Agile Model Driven Development (AMDD) is the application of AM to the software design and development process. Figure 1 shows that with AMDD, you do just enough high-level modeling at the beginning of a project to understand the scope and potential architecture of the system, and then during development iterations you take a just-in-time (JIT) model-storming approach where you model for several minutes as a precursor to several hours of coding. Extreme programmers (XPers) would call model-storming sessions stand-up design sessions or customer Q&A sessions.

Figure 1: The AMDD Lifecycle.

The Use of Agile Modeling (AMDD) for Achieving Assurance. The goal of AMDD is to provide a structure to the design aspect of agile development. There are four AM practices that have strong assurance benefits:

Conclusion

Assurance does not conflict with agility—it merely challenges it. For those applications that need it, a high-assurance process can be employed that retains the agility that the business needs to be able to respond to market changes and organizational changes. Explicit steps must be taken to ensure that the required assurance will be provided by the software application, so that it holds up under the stresses of high-visibility, high-usage, and mission-critical dependence that is characteristic of so many of today's Internet-facing applications in large organizations.

Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.