One Piece at a Time

Agile adherents claim that they can deliver working software at the end of a project's first week! They also claim that their software will always fulfill the highest-priority needs as defined by their stakeholders. Better yet, they do so even when they let stakeholders change their minds, add new requirements, rework them, or reprioritize them throughout the project. Agile practitioners work this way because they want regular feedback from their stakeholders and want to deliver the highest value possible. But how do they actually accomplish these worthy goals?

[click for larger image] Iterations and Releases Agile adherents deliver development releases into some form of demo environment at the end of each iteration and only occasionally release their system into production.

In my experience, an iteration should be no more than four weeks in length; otherwise you run the risk of working in a "mini waterfall manner. Although mini waterfalls are far less risky than traditional "big waterfalls, iterative approaches are even less dangerous. (See Chapter 6 of Craig Larman's Agile and Iterative Development: A Manager's Guide [Addison-Wesley, 2003] for a detailed discussion.)

It's easy to say that iterations should be short, but it's hard to do in organizations transitioning from traditional to evolutionary development. To overcome this challenge, I usually work with a group to determine an iteration length that they'll be just barely comfortable with, and then reduce iterations as they gain experience. Last year, I worked with a company that had 12-week iterations. To reduce them to eight weeks, we changed the review process from formal to informal, cut some paperwork, and co-located the entire team. During the next iteration, I convinced them to model iteratively, combining their requirements, architecture and design reviews into a single model review, which enabled them to reduce the third through fifth iterations to six weeks each. Once they saw that the world didn't end and recognized that the mid-iteration review served no purpose other than making the stakeholders happy, they chose to review the models at each iteration's end with the working software. The iterations were cut to five weeks and then four. By reducing the paperwork and bureaucracy typical of serial development, the team delivered roughly the same amount of functionality in four weeks as they did previously in 12.

Playing in the Sand

To incrementally release an application, you need a supportive development environment. The usual approach is to set up a collection of sandboxes—technical environments whose scope is well defined and respected by the people working in them. "Supportive Environment depicts each sandbox, the deployment effort between them, and the flow of defect reports. There are five types of sandboxes:

1. Development is the working environment for individual developers or programming pairs. In this sandbox, the developer/pair works in technical seclusion from the rest of the project team, enabling them to make and validate changes without having to worry about adversely affecting their teammates. This sandbox includes the tools required to develop and integrate the team's code, including a version of the database in which the developers can test their work.
2. The Project Integration sandbox is shared by members of a single project team. This environment is often referred to as a build environment or a build box. Developers promote their changed code to this environment, integrate it with the code written by the rest of the team, test it, and commit it into their configuration management system.
3. You deploy working systems into the Demo sandbox, the environment where your project stakeholders perform acceptance testing on the system. Minimally, your system should be deployed into this environment at each iteration's end, although interim releases are also possible.
4. The Test/QA sandbox is shared by several project teams and is often controlled by your testing/QA group. This environment is often referred to as a pre-production sandbox, a system testing area or simply a staging area. This environment simulates your actual production environment, so you can test your application in conjunction with other applications.
5. The Production sandbox is the actual environment in which your system will run after deployment.

[click for larger image] Supportive Environment Here is a depiction of each sandbox, the deployment effort between them, and the flow of defect reports.

Note that your organization may take a slightly different approach. Smaller organizations may choose to combine the Demo and Test/QA sandboxes. Larger organizations may choose to have various levels of Test/QA sandboxes; for example, one at the division level and one at the enterprise level. The important thing is that you separate the various environments and have an accepted way to promote applications among them.

Agile Deployment

Given the opportunity, I prefer to create builds in my own workspace several times a day, within my team workspace at least daily, into the pre-production test environment at each iteration's end, and into production anywhere between quarterly and yearly. Unfortunately, it's of little value to deliver working software on a weekly basis if it must go through a three-month pre-production testing process each time. With agility, you can reduce the time invested in pre-production testing efforts and thereby speed deployment. How is this done? Agile adherents produce high-quality artifacts and have the regression test suite to prove that their system actually works. This higher level of quality indicates that you'll be able to do less testing during system deployment. And once you get really good, you may find that you can deploy working software into production on a weekly basis—I've seen it done at a large financial institution and a dot-com. Yes, it's possible—regardless of what traditionalists may tell you.