Extreme Lessons Learned

Think you've got it tough on your current project? What if you were assigned to deliver a system with roughly 25 HTML pages, many of them interactive, with a team of developers who had never worked together before, most of whom had no experience in the selected technology, were not familiar with the problem domain, were not available full time, had no clear reporting structure, and were working for free? Did I mention that we only had three days in which to deliver a working system that was to be presented to our users in front of thousands of our peers on the fourth day? Is that "eXtreme" enough for you? Welcome to the first SD Project, developed at the Software Development East 2000 (www.sdexpo.com) conference, held the last week of October 2000 in Washington D.C.

This year the conference decided to have its speakers and attendees develop a system for a nonprofit organization—in this case, the redeployment of the Hydrocephalus Association's Web site (www.hydroassoc.org). The effort was led by Ken Pugh, of Pugh-Killeen Associates (www.pughkilleen.com), one of the speakers at the conference, and organized by Shabnam Malek, Conference Manager for the Software Development Media Group. I'm proud to say that the effort was successful, although not without a few hiccups. By the end, we all (re)learned several interesting lessons about software development.

The Initiation Phase
The project began several weeks prior to the conference; Shabnam lined up VA Linux (www.valinux.com) to provide equipment and development tools for the team. Ken developed an initial technical specification—in coordination with Jim Kranz, a board member of the Hydrocephalus Association—and then formulated a list of development tasks and the skills required to complete those tasks. Shabnam also asked the conference speakers, via e-mail, to volunteer. Ken distributed his documents to the volunteers, so we could familiarize ourselves with the effort before arriving at the conference. This initial preparation by a few people enabled the rest of the team to hit the ground running.

The core of the development team got together on Sunday evening to discuss the project, to understand the scope, and simply to meet everyone else. This was effectively a project kick-off meeting held over dinner. At dinner, we quickly realized that the composition of the team wasn't ideal; there were no Perl programmers among us, and few of us had Linux experience. In short, we were top heavy with managers and analysts, and desperately short of programmers. This situation is typical of many projects: Senior management had effectively made a technical decision to use Linux and Perl without first consulting the development team. Luckily, we actually required Perl as it was the only CGI scripting language supported by all the potential host systems, and Linux is a great operating system. Also, we likely had several people more than we needed at the beginning of the project; we had front-loaded the project with staff, but in the end, we were glad to have everyone on board.

Luck was on our side when Lorah McArdle, copy editor at Software Development magazine, hunted down some code for us and sent Ken the URLs for some sites with Perl code on them. Ken then downloaded an appropriate example after selecting it based on its description. With a little bit of investigation, you can often find reusable artifacts appropriate for your project team.

On Monday morning, we began by defining the initial scope and planning the project, and it quickly became obvious that we had more to do than we could handle—we were working on an unrealistic schedule. Typical software project. We had suspected this the night before, but it wasn't confirmed until we actually started planning. It was about this time that Shabnam, effectively senior management for the project, popped in to see how things were going. She was eager to know if we would have the system running in time for the presentation she had scheduled on Thursday. Wanting to soothe her fears, I told her that everything was on track and there was nothing to worry about: I lied and she went away happy. I usually want to be honest with everyone involved in a project, but by this time, the important factors (schedule, team, environment) were carved in stone. Worrying Shabnam would only slow us down.

Monday was a tutorial day, so the core team focused on building the foundation upon which the other volunteers would operate. We developed a list of good things to know about the project (a project overview, our goals, the site URL and our working directories) and posted it on the walls. This would help us to integrate new people into the project easily, an important goal considering people would join the project for only a few hours at a time. We also defined naming standards for the HTML fields, and we adopted some Perl coding conventions we found online because we felt that consistency was important as someone other than us would have to maintain the system over time.

We then developed a user interface flow model, also called a navigation map, which showed the HTML pages and how to traverse between them. We posted it on the wall, so everyone could see it and change it as the project progressed. The model was a collection of sheets of paper connected by masking tape, which indicated the pages that you could navigate to. Each piece of paper revealed the purpose of the HTML page, its name (for example, donations .html), whether it was an input form or static text, and its current development status. This model was simple; we clearly didn't use a CASE tool, yet it was vital to our project's success because it was user interface intensive (it's a Web site). User interface flow diagrams are not yet part of the Unified Modeling Language (UML), but you can read more about them and their use in my forthcoming book, The Object Primer, Second Edition (Cambridge University Press, 2001), and in Software For Use (Larry L. Constantine and Lucy Lockwood, ACM Press, 1999).

We tried a similar approach with the initial project plan but had much less success. We taped some flip chart paper to the wall and put sticky notes on the paper, one note for each task that we needed to accomplish. Although this effort allowed us to get a better handle on what needed to be done, it was abandoned on Tuesday once we lost the person leading the planning effort. The project plan was kept as simple as possible, but the reality was that the developers were focused on writing software, not updating a management document. They quickly migrated to an even simpler task list that they maintained throughout the effort. In short, the important artifacts of one set of team members were meaningless to others and therefore not kept current; you need to focus on developing and maintaining only a minimal set of artifacts.

The Elaboration Phase
On Monday, we set the architecture for the system and proved that it worked by developing an end-to-end prototype (a data input form). This was important for several reasons: It familiarized the team with the technology they were using, it provided working examples that people could copy and modify, and it assured both project and senior management that the project was realistic.

In parallel, we developed sample HTML forms to work from. One form set the general layout and color scheme for each page, something that was copied extensively once we finalized the Web site's look and feel later in the project (in the middle of the construction phase). Another form provided examples of input fields for common information, such as a person's name and address, which was needed on several pages. This helped set the look and feel of the input form pages, enforced the naming conventions that had been set previously, and provided a reusable asset that could be applied throughout the project. Our construction efforts benefited dramatically by our decision to identify and develop common reusable assets early in the project.

The Construction Phase
With the basic architecture, guidelines and code examples in place, we were able to start the Construction phase on Tuesday morning. We luckily had our own workroom that contained all the development workstations and provided enough wall space to post our models and plans. Had the team been separated from one another, a typical situation that most developers face with cubicles, we would not have been able to produce as much software that quickly. We also had the benefit of our user, Jim Kranz, being actively involved with the project. He edited some of the HTML pages, particularly the ones that were static text, and reviewed the effort as it progressed. He also helped improve and then finalize the site's look and feel, enabling the team to apply that style to all the HTML pages. He naturally assisted in explaining the goals of the system and provided insight into the requirements. It was easy for our user to become an active and productive member of the team, once we found a productive role for him.

On Wednesday, a Perl expert became involved with the project for a short time—until that point, the team had been learning as they went and were struggling. The expert reviewed some of the source code and provided critical insights that got the programmers going in the right direction. With a small investment of time, the team was able to effectively leverage the knowledge of this expert tenfold.

That evening, disaster was narrowly averted. Due to miscommunication, a manager at VA Linux sent someone to pick up our equipment on Wednesday evening. They thought that we only needed the equipment until Thursday morning, which was true because the presentation was early Thursday afternoon; however, someone had translated that to mean they could pick everything up Wednesday night. Fortunately, some of the team were still there (we were just about to leave for dinner), and we were able to set everything straight.

The near disaster only proved one of my core beliefs: Our industry has such a dismal success rate, in part, because it's so easy to kill a project with one bad decision. In this case, the decision maker was far away from the situation, and the team he had sent didn't have sufficient knowledge to make the right call, forcing people near the situation (the project team) to override the original decision. Had we not been able to prevent VA Linux from taking the equipment, we very likely would not have been able to deliver in time for the presentation on Thursday (I doubt everything was fully backed up).

The Transition Phase
The presentation occurred on Thursday as originally planned, even though the system had not yet been deployed. The users had not yet identified an ISP to host the new version of the site, and, therefore, we couldn't deploy and test the site until we knew the exact operational environment. Although Perl is a cross-platform language, we still needed to validate that our system would work on the chosen platform.

The Production Phase
At Software Development West 2001 (www.sdexpo.com), to be held the second week of April in San Jose, California, a group of volunteers will develop an updated version of the site. By that time, the system will have operated for several months and our users will have identified and prioritized new requirements. Time will tell. I look forward to sharing a new set of lessons learned with you after SD West.