Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Channels ▼
RSS

Data Meets the Lifecycle


Data Meets the Lifecycle

What do leading software development processes such as Extreme Programming, Feature-Driven Development, the Rational Unified Process, the Enterprise Unified Process, the Dynamic Systems Development Method, Agile Modeling and Scrum have in common? They're all iterative and incremental: They take what's called an evolutionary approach to development. In fact, the only popular methodology that doesn't take this approach is IEEE 12207—and the IEEE is now in the process of defining a "standard" evolutionary lifecycle. It's clear that evolutionary development is becoming the industry norm.

Although most object developers are in the know, the concept is new for many data professionals. Clearly, we need to find techniques that support evolutionary database development. Is this possible? Absolutely.

Agile DBAs

At Software Development East 2001, independent consultant Peter Schuh first introduced the role of the agile DBA, who performs traditional DBA tasks such as database setup, performance tuning, physical data modeling, database development and developer support—but in an agile, evolutionary manner.

My vision of the agile DBA is slightly different than Schuh's—he narrowed in on a single team, whereas I examine the entire enterprise. The agile DBA works side by side with application developers, typically supporting a single large team or several small teams, as necessary. The agile DBA can often be responsible or co-responsible for several data sources. In large organizations, the agile DBA must also coordinate with enterprise architects and enterprise administrators to achieve mutual goals.

To become more agile, traditional DBAs must tackle three major hurdles. First, they must focus on working closely with others and improving communication. This can be accomplished through colocation with development teams and by favoring face-to-face discussion over electronic communication or documentation. Second, traditional DBAs must recognize that it's no longer necessary to work in a serial manner; they can successfully develop software without creating (nearly) complete data models up front. Third, they must adopt techniques that enable them to work in an evolutionary manner—techniques that include taking an agile modeling-driven approach, database refactoring, mapping nonrelational technology to relational databases, evolutionary performance tuning and test-driven development (TDD).

Rethinking Modeling

Modeling is a critical skill for all IT professionals, including agile DBAs. Unfortunately, this opinion isn't prevalent, probably because most developers' first exposure to modeling is often dysfunctional—many people have been turned off of modeling by Big Design Up-Front (BDUF) approaches, which demand significant amounts of documentation before coding begins or "documentation after the fact" efforts following a system's release. In contrast, Agile Model-Driven Development (AMDD) offers a minimalist alternative: Agile models—often simple sketches—that are "just barely good enough" are created to help you think things through before you build them.

AMDD differs from traditional Model-Driven Development (MDD), exemplified by the Object Management Group (OMG)'s Model-Driven Architecture (MDA) standard (www.omg.org) in that it doesn't require you to create highly detailed, formal models. Instead, AMDD is a streamlined development approach that reflects agile software development principles, providing a way to create the artifacts (such as physical data models) that are critical to the success of DBAs. The collaborative environment fostered by AMDD promotes communication and cooperation among everyone involved on your project. This helps break down some of the traditional barriers between groups in your organization and motivates all developers to learn and apply the wide range of artifacts required to create modern software—there's more to modeling than data models. (Visit for more information about AMDD.)

You're Lost Without a Map

Modern software development projects use a variety of technologies—objects, relational databases and XML, as a start—and agile DBAs need to understand how to use them together effectively. Because these technologies are based on different paradigms, there's an impedance mismatch among the three technologies. Luckily, you can easily overcome this barrier by understanding how to map the three schemas to each other, and how to implement those mappings. At Agile Data, you can find several essays detailing these very issues.

It's All About Change

In his book Refactoring (Addison-Wesley, 1999), Martin Fowler introduced the concept of refactoring: small improvements to your source code that enhance its design without adding new functionality. Similarly, a database refactoring is a small improvement to your database schema that doesn't change its functional or informational semantics. Like code refactoring, database refactoring enables you to evolve your design over time, thus helping agile DBAs meet the new needs of their stakeholders. Database refactoring supports evolutionary database design as well as the incremental improvement of existing, legacy database schemas.

Database refactorings are a way to normalize your physical database schema after the fact. Ideally, you should try to get your database schema right to begin with—which is why AMDD is important. However, this doesn't always happen, and even if it does, new or changed requirements may be lurking in the wings. For more information about database refactoring, take a look at my February 2003 column, "Ten Steps to a Robust DB," and visit the Catalog of Database Refactorings for a list of known database refactorings.

Vroom, Vroom!

Because both your object schema and your database schema evolve over time, your application's performance will vary. In the past, performance tuning was often left until late in the lifecycle, when most of the system was in place. Agile teams, however, produce working software in an incremental manner, at the end of each iteration or cycle, which are often as short as a week or two. Thus, performance tuning should occur throughout development in an evolutionary manner.

Because modern developers typically work with multiple technologies, performance tuning becomes more complex. It isn't sufficient to tune just your database—you'll also find that you need to tune your object schema and your XML schemas as well as the mappings between them.

Test Before You Code

Test-driven development (TDD) is a programming approach in which you write a new test, watch it fail, and then write the little bit of functional code required to ensure that it passes (see my June 2003 column, "Extreme Testing," for more information). This incremental method, popular with agile developers, can also be applied to database development. Why not write a test before making a change to your database schema? Why not make the change, run the tests and refactor your schema as required? Although it's now in its nascent stage, I anticipate that TDD will be adopted by agile DBAs over the next few years.

TDD and AMDD go hand in hand. AMMD lets you create models with your project stakeholders to help explore their requirements, and then adequately explore those requirements in architectural and design models (often, simple sketches). TDD should be used as a critical part of your coding efforts to ensure that you create clean, working code. The result? A high-quality, working system that meets the actual needs of your project stakeholders.

Going Beyond Techniques

Successful database construction requires more than shiny new techniques—you also need tool support. Many agile teams work with the easy-to-use Prevayler, an alternative to relational databases, at the beginning of a project, and sometimes even deploy with it because of its superior performance. Some testing tools do exist; in particular, DBUnit and utPLSQL, although they both need some work. There's a wide variety of persistence frameworks, such as Castor and TopLink, that include mapping tools. I know of at least one database refactoring tool, and potentially another, that are in development and should be available sometime in 2004. The tool situation isn't adequate yet, but it's improving.

Data professionals need to adopt new ways to work if they want to remain relevant to modern software development teams. To help you move beyond this overview into the nitty-gritty, my new book, Agile Database Techniques: Effective Strategies for the Agile Software Developer (Wiley, Sept. 2003), presents a much more thorough look at this topic.

Scott W. Ambler is a senior consultant with Ronin International Inc. His latest book is the forthcoming Agile Database Techniques: Effective Strategies for the Agile Software Developer (Wiley, 2003).


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.