Continuous Integration and beyond...

Thomas is a manager with Deloitte. He is currently working on the firm's Commonwealth of Pennsylvania account and is located in Harrisburg, PA. Thomas can be reached at thbeckdeloitte.com.

A common practice at Microsoft and some other shrink-wrap software companies is the "daily build and smoke test" process. Every file is compiled, linked, and combined into an executable program every day, and the program is then put through a "smoke test," a relatively simple check to see whether the product "smokes" when it runs.
—Steve McConnell
"Best Practices: Daily Build and Smoke Test"
IEEE Software, July 1996

While unfamiliar to many developers, Continuous Integration is not a new concept at Microsoft, as is evident by Steve McConnell's observation. Consequently, in this second installment of a two-part article (see DDJ, August 2004 for Part I), I examine the setup of a version-control system and configuration of a Continuous Integration tool that runs off of the version-control system in the .NET environment. I then integrate these tools into the build process to test for conformance with the Microsoft .NET Framework Design Guidelines, and finally address the issue of code-coverage testing.

CVSNT: Team Builds & Source-Code Control

A source-code control system should be in place for all team development efforts. As a matter of fact, this should be one of the first things that is put in place as part of an organized development effort. The reason I've waited until now to introduce version-control mechanisms in this article is twofold:

• Version control is not critical to understanding the NAnt, NUnit, and NDoc examples in the first part of this two-part article.
• Version control is a prerequisite for the next topic of this article—Continuous Integration with CruiseConrol.NET. Both NAnt and CruiseControl.NET integrate well with a variety of version-control systems. However, I use Concurrent Versions Systems (CVS), an open-source version-control system that runs on Linux and Windows.

CVSNT is the Windows port of CVS (http://www.cvsnt.org/wiki/). CVS can be accessed from the command line using your IDE (both #Develop and Eclipse support CVS) or one of several popular open-source tools. For example, TortoiseCVS (http://www.tortoisecvs.org/) is a Windows plug-in that lets you access CVS capabilities from within the Windows File Explorer. Likewise, WinCVS (http://www.wincvs.org/) is similar in appearance to Visual Source Safe's user interface. When you have completed CVSNT setup, follow these steps to enter your code into version control:

1. Use the CVSNT Control Panel to add a repository named "Testing" underneath the default CVS directory.
2. Make a new module (/5-CVS, in this case) using the Password Server (:pserver) protocol within the repository that you created above. This creates a corresponding /5-CVS folder underneath your module folder. Figure 2 illustrates what this step would look like using TortoiseCVS (Figure 1 appeared in Part I).
3. Add all the contents of the root build directory to the CVS repository. This creates a number of additional CVS directories in your file structure. You should only be checking in source code, not the results of your build.
4. When you have added the files, you need to do an initial commit of all of the files to CVS. Your code is now in version control and you can proceed to automatically check it in/out using the build program I present here.

The changes to the build file to accommodate CVS are relatively minor; see Listing Six (Listings One through Five were presented in Part I). The goal of the changes is to make sure that the most current code is available prior to the build. The build-file changes amount to the addition of the update_source target, which contains the cvs-update NAnt task to update the build folder with the most current versions of our source code from CVS.

When your build runs, notice that check-ins are occurring from CVS. You can test this by deleting some of the source files from your local directory and confirming that they are replaced with the versions from CVS prior to the build. The next step is to automate source control access with a continuous integration tool.

CruiseControl.NET & Continuous Integration

CruiseControl.NET is an open-source tool from SourceForge designed to meld several other open-source building and testing tools and completely automate the build process (http://sourceforge.net/ projects/ccnet/ and http://confluence.pulic .thoughtworks.org/display/CCNet/Download/). In fact, the goal of Continuous Integration tools such as CruiseControl.NET is to automate the reaction to build events (such as source-code modifications). In Figure 3, changes to the project's source code trigger a build. This build causes the build site to be updated and, if necessary, results in e-mail notification of parties responsible for correcting unit testing errors. Coordination of these activities is the responsibility of CruiseControl.

The source code for this section of the article is set up similarly to the source code to the previous example with one exception, namely that you are using two build files instead of one. The first build file (bootstrap) is responsible for updating the code on the build server from CVS, then invoking the main build file (CCNet). The main build file performs all the functionality of the build file from the previous section, with the exception of the CVS access now performed by bootstrap.

Perform the initial addition and commit of the necessary files to a new CVS module, 6-CCNet, then perform an initial build by invoking the bootstrap build file. The first step in getting CruiseControl.NET up and running is to create a ccnet.config XML file that drives CruiseControl. This file controls the frequency of the integration cycles, defines the location of the version-control system and NAnt, details what type of (if any) e-mail notifications should be associated with the build, and controls the XML logging that is used as the basis for the build web site. An example ccnet.config file is available electronically; see "Resource Center," page 5. The file most likely needs to be tailored to accommodate the locations of specific files and programs on your filesystem. When you have the ccnet.config file in place, you can kick off CruiseControl by running the StartCCNet.bat job from the command line. The job should initiate an initial build and then continuously poll CVS until a change is made to the source code, at which point a new build will be started. Setting up the CruiseControl.NET web site is as simple as mapping an IIS virtual directory to the \web subfolder of your CCNet folder. Your continuous integration process is set up and ready to go.

FxCop and .NET Framework Design Guidelines

With the Continuous Integration process up and ready to go, it's worth introducing a valuable tool that plugs right into CruiseControl. FxCop (http://www.gotdotnet.com/team/fxcop/) is an application that checks .NET managed code assemblies for conformance with the Microsoft .NET Framework Design Guidelines (http:// msdn.microsoft.com/library/default.asp?url=/library/en-us/cpgenref/html/ cpconnetframeworkdesignguidelines.asp). The FxCop tool tests for conformance in:

• Library design.
• Localization.
• Naming conventions.
• Performance.
• Security.

Once FxCop is installed, you need to perform two steps:

1. Set up an FxCop task in the NAnt build file (see Listing Seven).
2. Merge the XML output produced by FxCop with the other input for CruiseControl (Listing Eight).

Step 1 invokes the FxCop command-line utility using the exec task. A custom NAnt task for FxCop is not yet available. The second step, preparing the FxCop XML output for consumption by CruiseControl is done in the ccnet.config file. In this case, the xmllogger's merging node (Step 2) is changed as in Listing Eight.

After making these changes, create a separate CVS folder for this build and redeploy the modified ccnet.config file. Once you've done this and kicked off your build with CruiseControl, you should be able to use the FxCop link on CruiseControl to view your conformance with the .NET Framework Design Guidelines as in Figure 4. You can then correct your code to comply with the guidelines. It is also possible to add custom rules to FxCop to accommodate any special guidelines in place in your particular organization.

Clover: How Thorough is Your Testing?

With the entire testing and integration process automated, you must consider how well all those unit tests that you're running are covering the different paths of execution within your code. To determine this, you need a tool that produces code-coverage metrics. Previously, such tools were usually only available as a part of large, usually fairly expensive, testing suites. Within the last few years, Clover, an inexpensive (though not open source) tool, has emerged that performs Java code-coverage testing, produces detailed reports and integrates with Ant (http://www.thecortex.net/clover/index.html). Originally developed as an internal tool at Cortex to support J2EE development, Clover is currently being ported to .NET and promises to be as powerful a tool for Continuous Integration in the .NET environment as it is in Java. A special limited 30-day licensed version of Clover is available electronically with the source code of this article. This Alpha release version of Clover has some limitations; for instance, only C# is currently supported. If the Java version is any indication, expect significant enhancements to this tool, its reporting functionality, and its integration with standard open-source tools as it moves towards a production-ready product.

The process of testing your code coverage using Clover.NET can be best explained using the example of the new Clover target added to our build file; see Listing Nine.

This target contains many familiar elements from previous sections and certain elements specific to Clover. Enumerated, the Clover target proceeds as follows:

1. Execute the program CloverInstr to create an instrumented version of the C# file and a Clover Coverage Database (.cdb). The instrumented C# file contains special "tracers" to facilitate linking back to the coverage database.
2. Compile the instrumented C# program making sure to include a reference to the Clover and CloverRuntime DLLs.
3. Compile and run the NUnit tests against the business object DLL created from the instrumented C# file.
4. Execute the Clover HtmlReporter program to produce the HTML code-coverage report. Figure 5 is a sample report.

It is worth paying attention to some of the Clover coverage features in Figure 5. The results include the code coverage for our class (BusObjCS.cs) as a percentage. If you had more than one class in your project, code-coverage results would also be summarized at the project level. The detailed coverage results for our class are below the coverage summary. Simple code highlighting indicates what was covered (line 23, six times in our tests), what wasn't covered (lines 18 and 24), and what need not be covered (lines 1-9). This type of visual detail lets you see exactly what is being tested, what's not being tested, and to achieve a quick overview of how thorough testing is across the different modules of a project. This type of feedback, combined with the unit test results, allows for the most effective refactoring of the code and unit tests.

Conclusion

Establishing a Continuous Integration environment using standard tools was a task that, just a few short years ago, was out of reach on a Microsoft development project. In the past several years, the number and quality of open-source build, testing, documentation, and integration tools supporting the .NET environment has increased drastically. Using the examples in this article, you should be able to weave together a Continuous Integration environment for your project that pays a hefty return on the comparatively minimal amount of time and resources that you need to invest.

DDJ

Listing Six

<target name="update_source">
    <cvs-update
        destination=".."
        cvsroot=":pserver:anonymouslocalhost:/testing"
        password="anonymous"
        module="5-CVS" />
</target>

<target name="cop" depends="test">
        <exec program="${fxcop.exe}" commandline="/f:${fxcop.src} 
        /o:${fxcop.out} /s" failonerror="false"/> 
</target>

<mergeFiles>
<file>c:\NETArticle\07FxCop\cs_build\BusObjTstCS.dll-   
     results.xml</file>
     <file>c:\NETArticle\07FxCop\cs_build\BusObjCS.ccnet-
        fxcop.xml</file>
</mergeFiles>

<target name="clover" depends="database">
  <mkdir dir="${clover.dir}"/>
  <copy file="c:\Program Files\ByteFX\ByteFX .Data 
        Provider 0.75\ByteFX.Data.dll" 
        tofile="${build.dir}\ByteFX.Data.dll" />
  <exec program="c:\program files\cenqua\clover.net\CloverInstr.exe" 
commandline="src\BusObjCS.cs -d ${clover.dir} -i ${clover.dir}\BusObjCS.cdb"/>
  <copy file="c:\program files\cenqua\clover.net\Clover.dll"tofile=
        "${clover.dir}\Clover.dll"/>
  <copy file="c:\program files\cenqua\clover.net\CloverRuntime.dll"
        tofile="${clover.dir}\CloverRuntime.dll"/> 
  <csc target="library" output="${clover.dir}\${basename}.dll"
        imports="System,System.Data,System.Data.SQLClient,
        System.Collections.Specialized,System.XML">
      <sources>
          <includes name="${clover.dir}\BusObjCS.cs"/>
      </sources>
      <references>
          <includes asis="true" name="System.dll"/>
          <includes asis="true" name="System.Data.dll"/>
          <includes asis="true" name="System.XML.dll"/>
    <includes name="${build.dir}\ByteFX.Data.dll"/>
          <includes name="${clover.dir}\Clover.dll"/>
          <includes name="${clover.dir}\CloverRuntime.dll"/>
      </references>
    </csc>
    <copy file="${nunit.dll}" tofile="${clover.dir}\nunit.framework.dll"/>
    <csc target="library" output="${clover.dir}\BusObjTstCS.dll"
        imports="System,System.Data,System.XML,BusObjCS,
        nunit.framework">
        <sources>
            <includes name="src\BusObjTstCS.cs"/>
        </sources>
        <references>
             <includes asis="true" name="System.dll"/>
             <includes asis="true" name="System.Data.dll"/>
             <includes asis="true" name="System.XML.dll"/>
             <includes name="${clover.dir}\Clover.dll"/>
             <includes name="${clover.dir}\BusObjCS.dll"/>
             <includes name="${clover.dir}\nunit.framework.dll"/>
         </references>
     </csc>
     <nunit2>
          <formatter type="Plain" />
          <test assemblyname="${clover.dir}\${testname}.dll"/>
     </nunit2>
     <exec program="c:\program files\cenqua\clover.net\
        HtmlReporter.exe" commandline="-i ${clover.dir}
        \BusObjCS.cdb -o report -t Test"/>
</target>