Apache Ant Best Practices

This chapter is from the book 

Project Support

All the Ant build.xml files that have been created so far have been quite simple. These files might suffice for building small Java projects, but the real world is not always so simple. You are probably wondering how Ant scales to projects with multiple components and a large code base. This section covers this issue and describes some of the capabilities and techniques that you can use for large or complicated projects.

Cumulative Targets

When creating a standard build file, you need to make sure that it can be used by all members of the development team. You don't want to have one set of build files for carrying out a Release Build or Integration Build and another set that the developers use to carry out their own Private Builds. Your aim should be to produce a single set of consolidated files that can be used by any member of the development team. However, there are often aspects of the complete build process that you might not want developers to carry out; baselining and deploying are typical examples. So how would you achieve this separation of concerns? Well, one way is to create an explicit cumulative target for carrying out Release or Integration Builds. In this target you specify how the Release or Integration Build is carried out as a sequence of individual targets.

There are two ways of creating a cumulative target. First, you can use the antcall task to invoke tasks in the same build file:

<target name="release" description="carry out Release Build">
    <antcall target="clearcase-pre" />
    <antcall target="update-buildinfo" />
    <antcall target="compile" />
    <antcall target="junit-all" />
    <antcall target="clearcase-post" />
    <antcall target="javadoc" />
</target>

However, the issue with this approach is that when using antcall, Ant reparses the build file and reruns all the targets that the called target depends on. In large projects this could slow down the build process. A better, and certainly faster, way is to use the depends attribute and create an empty target:

<target name="release" description="carry out Release Build" depends="clearcase-pre,update-buildinfo, compile,junit-all,clearcase-post,javadoc">
</target>

This example, although slightly less readable, does not reparse the complete Ant build file.

Reusable Routines

If you want to reduce the potential for duplication in your build files, you can reuse targets using the antcall target; you can also specify the value of attributes to be passed to the new target. For example, suppose you wanted to create a reusable target for checking ClearCase files in or out; you could create a new target called clearcase-op:

<target name="clearcase-op">
    <ca:ccexec>
        <arg value="${op.type}"/>
        <arg value="-nc"/>

        <arg value="${op.element}"/>
    </ca:ccexec>
</target>

You could subsequently call this target as follows:

<antcall target="clearcase-op">
    <param name="op.type"    value="checkout"/>
    <param name="op.element" value="${file.build.info}"/>
</antcall>

This is a good start. However, beginning with Ant 1.6, a better and quicker way of creating reusable routines is to use the macrodef task. For example, you could create a reusable macro for any ClearCase command as follows:

<macrodef name="clearcase-op">
    <attribute name="type"/>
    <attribute name="element"/>
    <attribute name="extra-args"  default=""/>
    <attribute name="comment"     default="automatically applied by clearcase-op"/>
    <sequential>
        <ca:ccexec>
            <arg value="@{type}"/>
            <arg line="-c '@{comment}' @{extra-args}"/>
            <arg value="@{element}"/>
        </ca:ccexec>
    </sequential>
</macrodef>

Then, to check out a file, you would simply use the following in your build.xml file:

<clearcase-op type="checkout" element="${file.build.info}"/>

This makes the build.xml file much more readable. Macros should be used in this way wherever you have a particular task being called repeatedly but with different parameters. If you wanted to, you could even create macros for every cleartool command, such as checkin or checkout. You might find it worthwhile to scan your build file to see if any additional opportunities exist for creating macros, typically where a set of operations is required to be carried out as an atomic operation. One example is the application of ClearCase labels or baselines. With Base ClearCase, to apply a label you create the label type, apply the label, and then promote it. A macro to achieve this could be created as follows:

<macrodef name="cc-apply-label">
    <attribute name="label"/>

    <attribute name="plevel"    default="BUILT"/>
    <attribute name="startloc"  default="."/>
    <sequential>
        <!-- create a new label type -->
        <ca:ccexec>
            <arg value="mklbtype"/>
            <arg value="@{label}"/>
        </ca:ccexec>
        <!-- apply the label -->
        <ca:ccexec>
            <arg value="mklabel"/>
            <arg value="-recurse"/>
            <arg value="@{label}"/>
            <arg value="@{startloc}"/>
        </ca:ccexec>
        <!-- apply the promotion level to the label -->
        <ca:ccexec>
            <arg value="mkattr"/>
            <arg value="PromotionLevel"/>
            <arg value="\"@{plevel}\""/>
            <arg value="lbtype:@{label}"/>
        </ca:ccexec>
    </sequential>
</macrodef>

Then, to apply a label, you simply use the following in your build.xml file:

<cc-apply-label label="RATLBANKMODEL_01_INT" plevel="BUILT"/>

UCM requires fewer operations to achieve a similar effect; to apply a baseline and promote it, you could create a macro as follows:

<macrodef name="cc-apply-bl">
    <attribute name="basename"/>
    <attribute name="component"/>
    <attribute name="plevel"    default="BUILT"/>
    <attribute name="pvob"      default="\RatlBankProjects"/>
    <attribute name="baseline" default="@{component}_@{basename}"/>
    <sequential>
        <!-- create a new baseline -->
        <ca:ccexec>
            <arg value="mkbl"/>
            <arg value="@{basename}"/>
        </ca:ccexec>
        <!-- promote the baseline -->
        <ca:ccexec>
            <arg value="chbl"/>
            <arg line="-level @{plevel}"/>
            <arg value="@{baseline}@@@{pvob}"/>
        </ca:ccexec>
    </sequential>
</macrodef>

Then, to apply a baseline you simply use the following in your build.xml file:

<cc-apply-bl basename="01_INT" plevel="BUILT"
 component="RATLBANKMODEL"/>

You can change these macros to conform to the exact way you want to create labels or baselines (for example, you might want to lock the label type too), but the point is that you have placed it in a macro, standardized your approach, and automated it.

Reusable Libraries

Maintaining a single large monolithic build.xml can be a challenge, especially with some of the eclectic syntax that Ant dictates. In most traditional programming languages, when you develop code you normally create libraries of common routines to hide some of this complexity. The preceding section mentioned macros; they obviously would be good candidates for placing in a separate library of routines. As of Ant 1.6, you can use the import task to include a library of routines (from the file standard-macros.xml), as in the following example:

<project name="test" default="test">
    <import file="standard-macros.xml"/>
    ...
</project>

One word of warning, though: the imported file must be a valid Ant build file, and it requires a project root element, just like any other build file. Listing 5.4 shows a common routines file containing the sample macro from the preceding section as well as the definition of some <patternset>s and a macro to execute a JUnit test suite.

Example 5.4. standard-macros.xml Reusable Macros File

<?xml version="1.0"?>
<project name="standard-macros" description="standard reuseable macros"
    xmlns:ca="antlib:net.sourceforge.clearantlib">
...

<!-- define a patternset for Java classes -->
<patternset id="java.sources">
    <include name="**/*.java"/>
    <exclude name="**/*Test*.java"/>
</patternset>

<!-- define a patternset for Test classes -->
<patternset id="test.sources">
    <include name="**/*Test*.java"/>
</patternset>

...

<!-- macro for executing a clearcase operation -->
<macrodef name="clearcase-op">
    <attribute name="type"/>
    <attribute name="element"/>
    <attribute name="extra-args"  default=""/>
    <attribute name="comment"     default="automatically applied by clearcase-op"/>
    <sequential>
        <ca:ccexec>
            <arg value="@{type}"/>
            <arg line="-c '@{comment}' @{extra-args}"/>
            <arg value="@{element}"/>
        </ca:ccexec>
    </sequential>
</macrodef>

...

<!-- macro for running a junit test suite -->
<macrodef name="project-junit">
    <attribute name="destdir" default="${dir.doc}"/>
    <attribute name="packagenames" default="${name.pkg.dirs}"/>
    <attribute name="classpathref"/>
    <element name="testclasses"/>
    <sequential>

        <junit printsummary="on" fork="no"
         haltonfailure="false"
         failureproperty="tests.failed"
         showoutput="true">
            <classpath refid="@{classpathref}"/>
            <formatter type="xml"/>
            <batchtest todir="@{destdir}">
                <testclasses/>
            </batchtest>
        </junit>
        <junitreport todir="@{destdir}">
            <fileset dir="@{destdir}">
                <include name="TEST-*.xml"/>
            </fileset>
            <report format="noframes" todir="@{destdir}"/>
        </junitreport>
        <fail if="tests.failed">
        One or more tests failed. Check the output...
        </fail>
    </sequential>
</macrodef>
...

</project>

You could also place common targets inside a similar XML file. For example, you could place standard invocations of the init, update-view, and junit-all targets in a file, as shown in Listing 5.5.

Example 5.5. standard-targets.xml Reusable Targets File

<?xml version="1.0"?>
<project name="standard-targets" description="standard reuseable targets" xmlns:ca="antlib:net.sourceforge.clearantlib">

<!-- create output directories -->
<target name="init" description="create directory structure">
    <mkdir dir="${dir.build}"/>
    <mkdir dir="${dir.dist}"/>
    <mkdir dir="${dir.doc}"/>
</target>

...

<!-- ClearCase snapshot view update -->
<target name="update-view" description="update ClearCase snapshot
 view">
    <ca:ccexec failonerror="false">
        <arg value="setcs"/>
        <arg value="-stream"/>
    </ca:ccexec>
    <ca:ccexec failonerror="false">
        <arg value="update"/>
        <arg value="-force"/>
        <arg line="-log NUL"/>
        <arg value=".\.."/>
    </ca:ccexec>
</target>

...

<!-- run all junit tests -->
<target name="junit-all" depends="compile" description="run all junit tests">
    <project-junit classpathref="project.classpath"
        destdir="${dir.build}"
        packagenames="${name.pkg.dirs}">
        <testclasses>
            <fileset dir="${dir.src}">
                <patternset refid="test.sources"/>
            </fileset>
        </testclasses>
    </project-junit>
</target>
...

</project>

If both the standard-macros.xml and standard-targets.xml files are placed under ClearCase control and labeled or baselined, you can simply pull in the baselined version for your build process.

Using these capabilities, you can significantly reduce the content of each project's build.xml. For example, with targets for compilation, testing, creating distribution archives, and so on in these reusable files, the complete build.xml for a project might look like Listing 5.6.

Example 5.6. build.xml Using Reusable Library Files

<?xml version="1.0"?>
<project name="RatlBankWeb" default="help" basedir=".">
    <property environment="env"/>
    <property name="dir.src"     value="JavaSource"/>
    <property name="dir.build"   value="WebContent/WEB-INF/classes"/>
    <property name="dir.dist"    value="dist"/>
    <property name="dir.junit"   value="build"/>
    <property name="dir.doc"     value="doc"/>
    <property name="dir.lib"     value="WebContent/WEB-INF/lib"/>

    <property file="build.properties"/>
    <property file="default.properties"/>

    <import file="${env.JAVATOOLS_HOME}/libs/standard-macros.xml" optional="false"/>
    
   <import file="${env.JAVATOOLS_HOME}/libs/standard-targets.xml" optional="false"/>

    <!-- define a classpath for use throughout the file -->
    <path id="project.classpath">
        <pathelement location="${dir.build}"/>
        <!-- include java libraries -->
        <fileset dir="${env.JAVA_HOME}/lib">
            <include name="tools.jar"/>
        </fileset>
    </path>

<!-- project override for junit test suite -->
<target name="junit-all" depends="compile" description="compile source code">
    <project-junit classpathref="project.classpath"
     destdir="${dir.junit}">
        <testclasses>
            <fileset dir="${dir.src}">
                <patternset refid="test.sources"/>
            </fileset>
        </testclasses>
    </project-junit>
</target>

</project>

Notice that there is no definition for the init, clean, or compile targets, because they are all inherited. You will also notice that this project includes an override for the junit-all target. Although in Ant properties are immutable, targets can be redefined. So even if you have a standard invocation of a target in a reusable library file, you can still override it if desired, as in this case.

Build File Chaining

Large projects tend to develop build files per component or subsystem and create a top level or controlling build file to chain them together. This can be achieved using the <subant> task working on the results of a <fileset> defining which build files to call, as in the following example:

<target name="build-all">
    <subant target="compile">
        <fileset dir="." include="*/build.xml"/>
    </subant>
</target>

In this example, any file called build.xml in a directory below the current directory is called, and the compile target is invoked. One word of warning is that the filesets are not guaranteed to be in any particular order; this means that if there were dependencies between build components, compilation problems could occur.

Conditional Execution

Ant was not really intended as a general-purpose scripting language, so it lacks some of the capabilities found in scripting tools such as Perl, Python, and Ruby. In particular, currently it has very limited support for conditional execution or processing. However, it has some capabilities that should be sufficient in most circumstances, as I will describe here.

First, Ant allows you to conditionally execute a target depending on whether a property is set. For example, suppose you had some specific tasks that were dependent on the version of ClearCase that was installed, such as either Full ClearCase or ClearCase LT. You could then set the cclt property to indicate that ClearCase LT was installed (with any value, but for readability purposes I will set it to true). This property can then be used with the if or unless target attributes:

<property name="cclt" value="true"/>

<target name="cc-lt-check" if="cclt">
    <echo>ClearCase LT is installed; dynamic views not
     supported</echo>
    <!-- update snapshot view -->
</target>

<target name="cc-check" unless="cclt">
    <echo>Full ClearCase is installed; running audited
     build</echo>
    <!-- audit build -->
</target>

In this example the contents of the target cc-lt-check are executed only if the cclt property has been set. Likewise, the target cc-check is executed only if the property has not been set.

Ant also has a general <condition> task that can evaluate the result of grouping multiple logical conditions. This task can be used to set a property if certain conditions are true. The conditions to check are specified as nested elements. For example, the following code checks to see if the developer is carrying out a debug build:

<target name="debug-check">
    <condition property="debug">
        <and>
            <available file="build.properties"/>
            <istrue value="${value.compile.debug}"/>
        </and>
    </condition>
</target>

<target name="debug-build" depends="debug-check"
 if="${debug}" >
    <echo>carrying out a developer debug build</echo>
</target>

In this example, two conditions are checked: one using the <available> condition to see whether a particular file exists (in this case, build.properties), and another using <istrue> to see whether a particular property has been set to true (in this case, value.compile.debug—our debug switch). The <and> element specifies that both of the conditions must return true for the debug property to be set. There are a number of available conditions, including all the usual logical operators. For more information on evaluating conditions, see the Apache Ant Manual or Matzke [Matzke03].

The clearantlib library that I discussed earlier also has a ClearCase condition task that you can use in your build process. This task can be used to check for the existence of ClearCase objects, as in the following example:

<property name="dir.cc" value="build-results"/>

<!-- check whether ClearCase build results directory already
 exists -->
<target name="dir-check">
    <condition property="dir.ccexists">
        <ca:ccavailable objselector="${dir.cc}"/>
    </condition>
</target>

<!-- create build results directory, if it doesn't already
 exist -->
<target name="init" depends="dir-check" unless="dir.ccexists">
    <echo>creating ClearCase directory ${dir.cc}</echo>
    ...
</target>

In this example, executing the target init first executes its dependent target dir-check. This target uses the clearantlib <ccavailable> task to check whether a particular directory (in this case specified by the property ${dir.cc}) exists and is under version control. If the directory does exist under version control, the property ${dir.ccexists} is set by the <condition> task, and the body of the init task is subsequently executed. The <ccavailable> task can be used in a number of ways, such as to check for the existence of labels, branches, or baselines. For more information on the task, see the man page that comes with the distribution.

Groovy Ant Scripting

If you are familiar with the capabilities of scripting languages such as Perl, Python, or Ruby, you will probably find Ant's support for conditions, loops, and expressions quite limited. If you find that you can't quite do what you want with Ant's native capabilities, you can use scripting languages from inside your Ant build scripts. Since this book is about Java development, perhaps the most relevant scripting language and the one that I recommend is Groovy (http://groovy.codehaus.org/). Groovy is defined on its Web site as an "agile dynamic language for the Java 2 Platform." Basically, this means that Groovy is tightly integrated with the Java platform, and that it actually uses the Java 2 application programming interface (API) as the mainstay of its own API, rather than reinventing some new API that everyone would have to learn. Groovy has also attained Java Community Process (JCP) acceptance (as JSR #241) and therefore has been formally accepted by the Java community as the preferred language for Java-based scripting. Perhaps the most important feature of Groovy for our purposes is the fact that it understands the Ant domain. For example, you could create the following Groovy script to compile some Java source code:

projSrcDir  = "src"
projDestDir = "build"
projLibDir  = "libs"
ant = new AntBuilder()
projClassPath = ant.path {
    fileset(dir: "${projLibDir}"){
        include(name: "*.jar")
    }
    pathelement(path: "${projDestDir}")
}
ant.mkdir(dir: projDestDir)
ant.echo("Classpath is ${projClassPath}")
ant.javac(srcdir: projSrcDir, destdir: projDestDir, classpath: projClassPath)

This example uses Groovy's AntBuilder class to execute a selection of Ant tasks that should be familiar from the preceding chapter—<mkdir> and <javac>. The example also constructs an Ant <fileset> called projClassPath to be used as the Java CLASSPATH for building against. This is a relatively simple example, but with Groovy's support for variables, loops, and classes, you can more freely define your build script. In fact, using this capability you could actually replace all your Ant build scripts with equivalent Groovy scripts. However, the caveat here is that CruiseControl does not currently support the execution of Groovy scripts, so you would have to schedule their automation using a different tool. More importantly, however, you might already have a significant investment in your own or third-party Ant scripts that you would prefer not to rewrite. The alternative in this case is to use Groovy scripting from inside your Ant build files.

To use Groovy inside your Ant build files, you can use the Groovy Ant task (http://groovy.codehaus.org/Groovy+Ant+Task). Using this task you can place a series of Groovy statements between <groovy> markup tags:

<groovy>
  curdate = new java.util.Date()
  println("It is currently ${curdate}")
</groovy>

This example simply prints the current date from your Ant build file; you can see how the standard Java API call to the Date class is being used. As a more complete example, let's look at how to do something more useful. For example, what if you wanted to check in your build results (your .jar files) to a release area in ClearCase? Well, you could use Ant, Groovy, and the ClearCase <ccexec> task that was created earlier to achieve this using the Ant build script shown in Listing 5.7.

Example 5.7. build.xml Groovy Scripting

<?xml version="1.0"?>
<project name="groovy-build" basedir="." default="main">
    <property name="dir.dist" value="dist"/>
    <property name="dir.rel" value="C:\Views\RatlBankModel_bld\RatlBankReleases"/>
    <property name="dir.ant" value=" C:\Views\RatlBankModel_bld\JavaTools\ant\lib"/>

    <!-- declare groovy task -->
    <taskdef name="groovy"
     classname="org.codehaus.groovy.ant.Groovy">
        <classpath>
            <path location="${dir.dist}/groovy-1.0-jsr-03.jar"/>
            <path location="${dir.dist}/asm-2.0.jar"/>
            <path location="${dir.dist}/antlr-2.7.5.jar"/>
        </classpath>
    </taskdef>

    <!-- declare ccexec task -->
    <taskdef name="ccexec"
     classname="net.sourceforge.clearantlib.ClearToolExec">
        <classpath>
            <path location="${dir.ant}/clearantlib.jar"/>
        </classpath>
    </taskdef>

<!--  execute some groovy commands -->
<target name="main">
    <groovy>
        // get the value of the Ant properties
        def dist = properties.get('dir.dist')
        def rel  = properties.get('dir.rel')

        // create a scanner of filesets for the dist directory
        scanner = ant.fileScanner {
            fileset(dir: dist) {
                include(name:"**/*.jar")
            }
        }

        // iterate over the fileset
        def dirco = false
        for (f in scanner) {
           def toFilename = rel + "\\" + f.name.tokenize("\\").pop()
           def toFile = new File (toFilename)

           if (toFile.exists()) {
              // if file already exists, update it
              ant.ccexec(failonerror:true) {
                  arg(line: "co -nc ${toFilename}")
              }
              ant.copy(todir: rel, file: f)
              ant.ccexec(failonerror:true) {
                  arg(line: "ci -nc ${toFilename}")
              }
           } else {
              // if file does not exist, check out directory
              if (!dirco) {
                  // check out directory
                  ant.ccexec(failonerror:true) {
                      arg(line: "co -nc ${rel}")
                  }
                  dirco = true
              }
              ant.copy(todir: rel, file: f)
              ant.ccexec(failonerror:true) {
                  arg(line: "mkelem -ci -nc ${toFilename}")
              }
           }

        }

        // check in directory if still checked out
        if (dirco) {
           ant.ccexec(failonerror:true) {
               arg(line: "ci -nc ${rel}")
           }
        }
    </groovy>
</target>

</project>

In this example, first Ant properties are defined for the build (${dir.dist}) and release (${dir.rel}) areas. Then two <taskdef> declarations are made—first for the <groovy> Ant task and then for our ClearCase <ccexec> task. Using the <taskdef> statement is simply another way of registering a prebuilt Ant Java class. Next in the main target, rather than using Ant tasks, Groovy script is used. First the Groovy script accesses the Ant properties defined earlier and stores them in some variables. It then constructs an Ant <fileset> scanner (fileScanner) to locate all the .jar files in the ${dir.dist} directory. The script then iterates over all the entries in this <fileset>, copies the files to the release area, and adds or updates them in ClearCase. The interesting thing to note here is that some logic is built into the script to check whether the .jar file being copied already exists in ClearCase. This determines the exact ClearCase commands that will be used. For example, if the file is new, the directory needs to be checked out and a ClearCase mkelem command executed to create a new element. Achieving a similar result in Ant is considerably more complex.

In practice, I recommend using Groovy scripting only when you find that you can't do the equivalent operation well or very easily with standard Ant tasks and capabilities. It is obviously another language to learn. However, if you are interested in Java development, build processes, and scripting, I believe this will be a worthwhile exercise. The fact that Groovy is already a Java standard should also increase its visibility and popularity. For more information on the Groovy language, visit the Groovy Web site at http://groovy.codehaus.org/.

Build Script Documentation

One of the main documentation tasks for any build script is making sure that each target has an appropriate description attribute defined. For example, an obvious description for the compile target would be

<target name="compile" description="compile all source code">

The main reason for including this level of documentation is so that you can use the Ant -projecthelp (or -p) option from the command line. This option prints a list of all the targets in the build file, along with their descriptions:

>ant -p
Buildfile: build.xml

   Main targets:
  
   clean          remove generated files
  
   compile        compile all source code
...

As you include more and more potential targets in your build file, this self-documenting feature can become very useful for any user who wants to execute it.