Enough of Processes: Let's Do Practices Part I

In the first installment of this two-part article, our authors examine the problems created by today's software development processes—and why we need to change.


March 12, 2007
URL:http://drdobbs.com/architecture-and-design/enough-of-processes-lets-do-practices-pa/198000264

Ivar is one of the "fathers" of components and component architecture, use cases, UML, and the Rational Unified Process. Pan-Wei and Ian are chief scientists at Ivar Jacobson Consulting. The authors can be contacted at www.ivarjacobson.com.


All modern software development processes try to help project teams conduct their work. While there are some important differences between them, the commonalities are far greater—and understandably, since the end goal of them all is to produce working software quickly and effectively. Thus, it doesn't matter which process you adopt as long as it is adaptable, extensible, and capable of absorbing good ideas, even if they arise from other processes.

To achieve this kind of flexibility, things need to change. The focus needs to shift from the definition of complete processes to the capture of reusable practices. Teams should be able to mix-and-match practices and ideas from many different sources to create effective ways of working.

In this new approach to process, which we call "EssWork," teams select practices, which are then composed and integrated seamlessly into their development environments to provide dynamic and contextualized guidance, active facilitation, and even help remove mundane tasks through clever automation.

In the first installment of this two-part article, we examine the issues facing the current generation of processes and show why we have all had enough of them. In the next installment, we present EssWork and show how it can help teams realize their investment in learning, developing, and documenting best practices.

The Current Age: Processes

Every project team needs to have some way of working together—a way that is effective and delivers quality results on time. While every project team works differently, the way in which they work is usually inspired by what experts—internal or external—advocate. These experts have their own ways of working—their own methods, their own strategies, their own approaches, their own processes. The experts then feel the need to publish their preferred way of working in as unique and individual a way as possible, packaging their knowledge up as product—typically as a branded process, a standalone reference, or even a process framework. There are several reasons why this is a problem.

The Problem of Denied Commonality

With denied commonality, each process has a few interesting gems, but they are embedded in a larger package of commonalities. This is because no individual is good at everything, and an individual is usually only an expert on a few things.

There are many different processes promoted within the software development industry. But are they really that different? If you look closely at their content, you find that there are more similarities than differences. The reasons for this are straightforward:

The bottom line is that the processes have more commonality than differences. However, this can be difficult to detect as the champions of each process dress the common stuff in new words and make it sound as if everything is new.

What Is a Process?

So what exactly do we mean by "process"? Process is one of those terms that are often used in our industry without any clear or consistent meaning.

In this article, we use the term "process" as an umbrella to denote a published description of a way of working. It is not a universal term because the agile camp does not use it as such. Nevertheless, in the context of this article, we can use the term "process" understandably.

Because everything in every new process looks new, it can be hard to really compare processes. It is even harder to mix and match—extract the gems and combine them with the gems from other processes.

The Problem of Completeness

In terms of completeness, each process definition—large or small—wants to describe a complete process. Where the authors set out to describe a single area of the process, they inevitably end up trying to describe them all, typically in a tightly coupled and homogeneous fashion.

When processes are published, their creators often find it hard to resist describing every aspect of software development. Even when they do, the methodologists who follow in their footsteps generally try to fill in the gaps. The end result is that, as time passes, every process strives to be complete. But is this a problem?

For methodologists, it doesn't seem to be a problem. They happily "borrow" ideas from one another, adding spin, and in some cases, getting it wrong. If you speak to people using the processes—those who do real software development—this is a significant problem. By striving for completeness, the processes end up as brittle, all-or-nothing propositions. This makes it hard for teams to adopt just the good parts of the process and identify truly new ideas and techniques.

The desire to provide a complete process also makes the process heavier as more and more information is added to cover all of the disciplines involved. As the process evolves, no one ever takes anything out because someone somewhere might need it some day (a day that never seems to come). If a process is successful, then the desire to keep it up to date and extend its market leads it to become even heavier as more information is added to expand its scope, add new techniques, and address new challenges. This leads to the need for organizations, practitioners, and methodologists to start trimming to get a lighter, more-focused process. We don't think this is the way to go. You shouldn't need to spend time deselecting the things you don't want. You should be able to start from a small, useful kernel of information, then add the guidance you need.

The Problem of Adopting a Complete Process

When it comes to adopting a complete process, each software team has its own way of working (explicit or tacit). Changing everything is silly, changing one thing may be smart.

Adopting a new process presents teams with many challenges, especially as they will already have their own way of working. This may be expressed explicitly as a documented process or it may be more tacit. Within the team's way of working, there are always good practices that they will want to continue using. Other areas of the process will be weaker and lead to the desire for change. The problem with the branded processes is that they do not address this reality and require the team to change everything just to get the few new things that they want.

The same is true for organizations where, in addition to the many available processes, they have their own in-house processes that have evolved whilst developing their software. Generally, they might like to improve these processes, but adopting another (branded) process would be too big a change. You should be able to add and drop practices as needed.

The Problem of Out-of-Sync Processes

What a team actually does never matches the adopted process. Teams improvise and invent more effective ways of doing things; they find they need to work out solutions to problems nobody thought of when the process was selected, and they never keep the process descriptions up to date.

Often, there are complaints when a team seems not to be following the defined process. However, these complaints miss the point. The point of software development is to develop good software, not to slavishly follow a predefined process.

If you have ever been involved in a process assessment, you know that what the team does rarely matches the official process. This is true of all processes. People use a mixture of ideas and practices from various sources, and although they might claim to be advocates of specific processes, few of them are actually following them to the letter.

This gap between the process as described and the process as applied is what we call the "project-process" gap. This gap causes problems for projects, teams, and organizations:

The process should be a description of what the team actually does, rather than a fictional description of what people think the team ought to be doing.

The Problem of Acquiring Knowledge

To learn about a project, people don't read process manuals or language specifications. This is because they want to apply processes, not read about them. Moreover, they want different levels of detail at different times.

Processes are not just textual guidelines for reading pleasure. They need to be helpful and available when people are actually developing software.

Currently, the majority of existing processes are published as books and static web sites—shared resources that practitioners are expected to sit down and read. And we know that practitioners do not "read" per se—they will look the odd thing up on the Internet, but not read long, scrolling pages of text. When they are actively engaged in developing software they want succinct, focused, unambiguous guidance that will immediately help them undertake their work, and not long explanations, anecdotes, and academic treatises that justify the techniques they are trying to apply.

A consequence of this is that it doesn't make sense to write a lot of long and lengthy process descriptions. Process documentation needs to be presented in a new way, one that provides the practitioners with the information they need when they need it; leaving the books and knowledgebases to publish the academic foundations that the processes draw upon. Let the processes facilitate the active development of the software and the application of the practices the team selects.

The Problem of Stupid Processes

Most of the time, the process doesn't actively help people do their jobs because:

Have you ever tried to find anything useful in a process whilst you are actually developing some software? Typically, the process just sits there as a passive knowledgebase waiting patiently for you to read it. It certainly doesn't interact with you and offer you appropriate and timely advice. It expects you to know exactly what you are doing and exactly what useful information it contains and exactly where it has hidden it.

When you eventually find something in the process, it is typically not that useful. What you really want is some active help in the task you are trying to do and all you get is more words to read.

Processes often provide detailed instructions, for example:

Wouldn't it be nice if, rather than just describing these things, the process could actually play an active role in applying them? Rather than describing how you should do these things, it could actually do some of them for you: finding things, checking things, and performing mundane and trivial tasks.

Without the ability to take an active role in helping people to develop software, the process will always be nothing more than a fancy textbook, and teams will continue to struggle to realize its benefits. Process adoption will continue to rely on the presence of full-time experts, mentors, and coaches who will spend a lot of their time helping with trivial work and answering the same questions over and over again—a particularly expensive way of bringing the process to life, and one, which unfortunately, doesn't scale.

What Do These Problems Mean?

Taken together, these problems are making it harder and harder for companies and teams to establish an effective way of working. The processes are becoming a barrier to change rather than an enabler. The solutions being produced don't really address the needs of development teams. A lot of good ideas are arising in the industry; however, other good ideas are being discarded in a vain attempt to be fashionable and to be seen as keeping up to date.

Each of the problems outlined leads to paradoxes and contradictory behavior. For example, given that people don't read processes, why do so many people write them? Agile methods have been designed to rely on tacit knowledge instead of explicit knowledge; still, agile experts spend much of their time writing books and processes.

Lots of people are making lots of noise. Every practice that appears is labeled as a "best" practice. Every tool is presented as a solution to all the problems. But are we really moving forward? The industry is not really getting any better at developing software, the problems are just getting moved around. There is no firm foundation of first principles and cracks are starting to appear. The longer you spend in the industry, the more and more the success of each new process looks like another case of the Emperor's New Clothes—another triumph of spin and marketing over innovation and substance. A lot of excitement and noise is generated, but if you look closely, there is very little new that is actually there.

There needs to be a better way to collect and share knowledge. If it only sits in the head of a few special people, it won't help us scale up. We need to break out of the patterns of the past and find new ways of capturing and sharing knowledge.

Imagine we can:

The problems of the past then go away and we can really start to improve the way that we develop software.

Next Month

Next month, we examine a new way of capturing and bringing practices to life—one that is not just an academic theory, but something that is already working and in place.

Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.