Maven build and release multiple interdependant projects

Question

I have some existing Maven projects.

They are:

1. aaa (plugin)
2. bbb (jar)
3. ccc (jar)
4. ddd (war)
5. eee (war)

Project ddd is for a customer and eee is for another

They are located in flat structure under workspace/ folder on disk, and the very same structure is inside svn repo.

This is the project's dependency hierarchy:

ddd (war)        
    aaa (plugin)
    bbb (jar)
    ccc (jar)
        bbb (jar)

eee (war)
    ccc (jar)
        bbb (jar)

When only wars are SNAPSHOTs there's no problem in building and releasing

otherwise, i.e.

ddd       1.1-SNAPSHOT (war)        
  aaa     2.0          (plugin)
  bbb     2.1-SNAPSHOT (jar)
  ccc     1.3-SNAPSHOT (jar)
    bbb   2.1-SNAPSHOT (jar)

I have to do, for building:

1. bbb> mvn install
2. ccc> mvn install
3. ddd> mvn compile

and for release:

1. bbb> mvn release:prepare release:perform
2. ccc> mvn versions:use-releases
3. ccc> svn ci -m "updated SNAPSHOT dependencies"
4. ccc> mvn release:prepare release:perform
5. ddd> mvn versions:use-releases
6. ddd> svn ci -m "updated SNAPSHOT dependencies"
7. ddd> mvn release:prepare release:perform

I tried with an aggregator, but

• on building, it compiles non SNAPSHOT dependencies (aaa 2.0 -> compiles aaa 2.1-SNAPSHOT)
• on releasing, it complains about scm, but I don't want the aggregator to be in svn

This is what I need:

• single command build
  • orders SNAPSHOT dependecies
  • installs (or deploy) each SNAPSHOT dependency
  • builds (compile or package or install...) root artifact
• single command release
  • orders SNAPSHOT dependencies
  • releases each SNAPSHOT dependency
  • releases root artifact

This is what I don't want:

• a batch build/release script
• put aggregators in svn
• version sharing among projects
• parent becomes an aggregator too

Is this possible?

Alternative best practices? (maybe Jenkins will help?)

Should I switch to Gradle??

---

update

i see that most of you confuse parent with aggregator, so i removed it from this question. however,

[...] You will often see projects that are both parents and aggregators. [...] However, although both POM projects, an aggregator project and a parent project are not one in the same and should not be confused. A POM project may be inherited from - but does not necessarily have - any modules that it aggregates. Conversely, a POM project may aggregate projects that do not inherit from it.

from A final note on Inheritance v. Aggregation

Answer 1

just got it in a hacky way... but I'm not satisfied.

If someone will provide a better way to perform a custom recursive release process on maven I'll be glad to change my acceptance!

parent pom:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.mycompany</groupId>
    <artifactId>test-release-parent</artifactId>
    <version>1-SNAPSHOT</version>
    <packaging>pom</packaging>

    <name>test-release-parent</name>
    <description>This is the parent pom to test release procedure</description>
    <inceptionYear>2014</inceptionYear>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <svn.repository>http://svn.mycompany.com/repo1</svn.repository>
        <web.projects>http://www2.mycompany.com/projects</web.projects>
    </properties>

    <scm>
        <developerConnection>scm:svn:${svn.repository}/${project.artifactId}</developerConnection>
        <url>${svn.repository}/${project.artifactId}</url>
    </scm>

    <distributionManagement>
        <repository>
            <id>ftp.mycompany.com</id>
            <name>mycompany Maven Repository</name>
            <url>ftp://ftp.mycompany.com/maven-repo</url>
        </repository>
    </distributionManagement>

    <repositories>
        <repository>
            <id>www2.mycompany.com</id>
            <name>mycompany Maven Repository</name>
            <url>http://www2.mycompany.com/maven-repo</url>
        </repository>
    </repositories>

    <pluginRepositories>
        <pluginRepository>
            <id>www2.mycompany.com</id>
            <name>mycompany Maven Repository</name>
            <url>http://www2.mycompany.com/maven-repo</url>
        </pluginRepository>
    </pluginRepositories>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>commons-io</groupId>
                <artifactId>commons-io</artifactId>
                <version>2.4</version>
            </dependency>
            <dependency>
                <groupId>commons-codec</groupId>
                <artifactId>commons-codec</artifactId>
                <version>1.9</version>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-scm-plugin</artifactId>
                    <version>1.9</version>
                </plugin>

                <plugin>
                    <groupId>org.codehaus.mojo</groupId>
                    <artifactId>versions-maven-plugin</artifactId>
                    <version>2.1</version>
                    <configuration>
                        <excludes>
                            <exclude>javax:javaee-api:*:*</exclude>
                            <exclude>org.eclipse.persistence:*:*:*</exclude>
                            <exclude>org.hibernate:hibernate-validator:*:*</exclude>
                        </excludes>
                        <rulesUri>http://www.mycompany.com/ruleset.xml</rulesUri>
                    </configuration>
                </plugin>

                <plugin>
                    <groupId>org.codehaus.gmaven</groupId>
                    <artifactId>gmaven-plugin</artifactId>
                    <version>1.5</version>
                </plugin>
            </plugins>
        </pluginManagement>

        <plugins>
            <plugin>
                <groupId>org.codehaus.gmaven</groupId>
                <artifactId>gmaven-plugin</artifactId>
                <configuration>
                    <source>
                        String releaseVersion = project.version.substring(0, project.version.indexOf("-"));

                        String nextVersion;
                        int index = releaseVersion.lastIndexOf(".");
                        if(index == -1) nextVersion = (releaseVersion.toInteger() + 1) + "-SNAPSHOT";
                        else
                        {
                            String prefix = releaseVersion.substring(0, index);
                            String suffix = releaseVersion.substring(index + 1);

                            nextVersion = prefix + "." + (suffix.toInteger() + 1) + "-SNAPSHOT";
                        }

                        ant.exec(failonerror: "true", dir: "${basedir}", executable: "cmd")
                        {
                            arg(value: "/c")
                            arg(value: "mvn")
                            arg(value: "validate")
                            arg(value: "-Prelease-align")
                            arg(value: "-Dversion.release=" + releaseVersion)
                            arg(value: "-Dversion.next=" + nextVersion)
                        }

                        ant.exec(failonerror: "true", dir: "${basedir}", executable: "cmd")
                        {
                            arg(value: "/c")
                            arg(value: "mvn")
                            arg(value: "initialize")
                            arg(value: "-Prelease-prepare")
                            arg(value: "-Dversion.release=" + releaseVersion)
                            arg(value: "-Dversion.next=" + nextVersion)
                        }

                        ant.exec(failonerror: "true", dir: "${basedir}", executable: "cmd")
                        {
                            arg(value: "/c")
                            arg(value: "mvn")
                            arg(value: "deploy")
                            arg(value: "-Prelease-perform")
                            arg(value: "-Dversion.release=" + releaseVersion)
                            arg(value: "-Dversion.next=" + nextVersion)
                        }
                    </source>
                </configuration>
            </plugin>
        </plugins>

        <extensions>
            <extension>
                <groupId>org.apache.maven.wagon</groupId>
                <artifactId>wagon-ftp</artifactId>
                <version>2.6</version>
            </extension>
        </extensions>
    </build>

    <profiles>
        <profile>
            <id>buildall</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.codehaus.gmaven</groupId>
                        <artifactId>gmaven-plugin</artifactId>
                        <executions>
                            <execution>
                                <phase>validate</phase>
                                <goals>
                                    <goal>execute</goal>
                                </goals>
                                <configuration>
                                    <source>
                                        for(d in project.dependencies)
                                        {
                                            if(d.groupId == "com.mycompany" &amp;&amp; d.version.endsWith("-SNAPSHOT"))
                                            {
                                                println "installing " + d
                                                ant.exec(failonerror: "true", dir: "${basedir}/../" + d.artifactId, executable: "cmd")
                                                {
                                                    arg(value: "/c")
                                                    arg(value: "mvn")
                                                    arg(value: "install")
                                                    arg(value: "-Pbuildall")
                                                }
                                            }
                                        }
                                    </source>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>

        <profile>
            <id>release-align</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.codehaus.mojo</groupId>
                        <artifactId>versions-maven-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>initial-updates</id>
                                <phase>validate</phase>
                                <goals>
                                    <goal>update-parent</goal>
                                    <goal>use-releases</goal>
                                    <goal>commit</goal>
                                </goals>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>

        <profile>
            <id>release-prepare</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.codehaus.gmaven</groupId>
                        <artifactId>gmaven-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>release-snapshots</id>
                                <phase>validate</phase>
                                <goals>
                                    <goal>execute</goal>
                                </goals>
                                <configuration>
                                    <source>
                                        if(project.parent != null &amp;&amp; project.parent.groupId == "com.mycompany" &amp;&amp; project.parent.version.endsWith("-SNAPSHOT"))
                                        {
                                            println "releasing " + project.parent

                                            String releaseVersion = project.parent.version.substring(0, project.parent.version.indexOf("-"));

                                            String nextVersion;
                                            int index = releaseVersion.lastIndexOf(".");
                                            if(index == -1) nextVersion = (releaseVersion.toInteger() + 1) + "-SNAPSHOT";
                                            else
                                            {
                                                String prefix = releaseVersion.substring(0, index);
                                                String suffix = releaseVersion.substring(index + 1);

                                                nextVersion = prefix + "." + (suffix.toInteger() + 1) + "-SNAPSHOT";
                                            }

                                            ant.exec(failonerror: "true", dir: "${basedir}/../" + project.parent.artifactId, executable: "cmd")
                                            {
                                                arg(value: "/c")
                                                arg(value: "mvn")
                                                arg(value: "validate")
                                                arg(value: "-Prelease-align")
                                                arg(value: "-Dversion.release=" + releaseVersion)
                                                arg(value: "-Dversion.next=" + nextVersion)
                                            }

                                            ant.exec(failonerror: "true", dir: "${basedir}/../" + project.parent.artifactId, executable: "cmd")
                                            {
                                                arg(value: "/c")
                                                arg(value: "mvn")
                                                arg(value: "initialize")
                                                arg(value: "-Prelease-prepare")
                                                arg(value: "-Dversion.release=" + releaseVersion)
                                                arg(value: "-Dversion.next=" + nextVersion)
                                            }

                                            ant.exec(failonerror: "true", dir: "${basedir}/../" + project.parent.artifactId, executable: "cmd")
                                            {
                                                arg(value: "/c")
                                                arg(value: "mvn")
                                                arg(value: "deploy")
                                                arg(value: "-Prelease-perform")
                                                arg(value: "-Dversion.release=" + releaseVersion)
                                                arg(value: "-Dversion.next=" + nextVersion)
                                            }
                                        }

                                        for(d in project.dependencies)
                                        {
                                            if(d.groupId == "com.mycompany" &amp;&amp; d.version.endsWith("-SNAPSHOT"))
                                            {
                                                println "releasing " + d

                                                String releaseVersion = d.version.substring(0, d.version.indexOf("-"));

                                                String nextVersion;
                                                int index = releaseVersion.lastIndexOf(".");
                                                if(index == -1) nextVersion = (releaseVersion.toInteger() + 1) + "-SNAPSHOT";
                                                else
                                                {
                                                    String prefix = releaseVersion.substring(0, index);
                                                    String suffix = releaseVersion.substring(index + 1);

                                                    nextVersion = prefix + "." + (suffix.toInteger() + 1) + "-SNAPSHOT";
                                                }

                                                ant.exec(failonerror: "true", dir: "${basedir}/../" + d.artifactId, executable: "cmd")
                                                {
                                                    arg(value: "/c")
                                                    arg(value: "mvn")
                                                    arg(value: "validate")
                                                    arg(value: "-Prelease-align")
                                                    arg(value: "-Dversion.release=" + releaseVersion)
                                                    arg(value: "-Dversion.next=" + nextVersion)
                                                }

                                                ant.exec(failonerror: "true", dir: "${basedir}/../" + d.artifactId, executable: "cmd")
                                                {
                                                    arg(value: "/c")
                                                    arg(value: "mvn")
                                                    arg(value: "initialize")
                                                    arg(value: "-Prelease-prepare")
                                                    arg(value: "-Dversion.release=" + releaseVersion)
                                                    arg(value: "-Dversion.next=" + nextVersion)
                                                }

                                                ant.exec(failonerror: "true", dir: "${basedir}/../" + d.artifactId, executable: "cmd")
                                                {
                                                    arg(value: "/c")
                                                    arg(value: "mvn")
                                                    arg(value: "deploy")
                                                    arg(value: "-Prelease-perform")
                                                    arg(value: "-Dversion.release=" + releaseVersion)
                                                    arg(value: "-Dversion.next=" + nextVersion)
                                                }
                                            }
                                        }
                                    </source>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                    <plugin>
                        <groupId>org.codehaus.mojo</groupId>
                        <artifactId>versions-maven-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>final-updates</id>
                                <phase>initialize</phase>
                                <goals>
                                    <goal>update-parent</goal>
                                    <goal>use-releases</goal>
                                    <goal>set</goal>
                                    <goal>commit</goal>
                                </goals>
                                <configuration>
                                    <newVersion>${version.release}</newVersion>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>

        <profile>
            <id>release-perform</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-deploy-plugin</artifactId>
                    </plugin>

                    <plugin>
                        <groupId>org.codehaus.mojo</groupId>
                        <artifactId>versions-maven-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>set-next-snapshot</id>
                                <phase>deploy</phase>
                                <goals>
                                    <goal>set</goal>
                                    <goal>commit</goal>
                                </goals>
                                <configuration>
                                    <newVersion>${version.next}</newVersion>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-scm-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>checkin-release</id>
                                <phase>verify</phase>
                                <goals>
                                    <goal>checkin</goal>
                                    <goal>tag</goal>
                                </goals>
                            </execution>
                            <execution>
                                <id>checkin-snapshot</id>
                                <phase>deploy</phase>
                                <goals>
                                    <goal>checkin</goal>
                                </goals>
                            </execution>
                        </executions>
                        <configuration>
                            <tag>${project.version}</tag>
                            <message>auto-generated by release process</message>
                        </configuration>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>
</project>

and this is a project pom that uses another project (flatten structure in workspace) as dependency

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <groupId>com.mycompany</groupId>
        <artifactId>test-release-parent</artifactId>
        <version>1-SNAPSHOT</version>
    </parent>

    <modelVersion>4.0.0</modelVersion>
    <artifactId>test-release-project</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>${project.artifactId}</name>
    <description>maven project to test a release process</description>
    <inceptionYear>2014</inceptionYear>
    <url>${web.projects}/${project.artifactId}</url>

    <repositories>
        <repository>
            <id>www2.mycompany.com</id>
            <name>mycompany Maven Repository</name>
            <url>http://www2.mycompany.com/maven-repo</url>
        </repository>
    </repositories>

    <scm>
        <developerConnection>scm:svn:${svn.repository}/${project.artifactId}/trunk</developerConnection>
        <url>${svn.repository}/${project.artifactId}/trunk</url>
    </scm>

    <dependencies>
        <dependency>
            <groupId>commons-io</groupId>
            <artifactId>commons-io</artifactId>
        </dependency>
        <dependency>
            <groupId>commons-codec</groupId>
            <artifactId>commons-codec</artifactId>
        </dependency>

        <dependency>
            <groupId>com.mycompany</groupId>
            <artifactId>test-release-dependency</artifactId>
            <version>1.0.0-SNAPSHOT</version>
        </dependency>
    </dependencies>
</project>

just run

mvn <some-phase> -Pbuildall

to execute <some-phase> on current project and install on all owned and referenced snapshots (parent and dependencies)

and

mvn groovy:execute

to perform a release an all owned and referenced snapshots

the idea behind:

• procedure release:
  1. update parent version if exists
  2. update snapshot dependencies if exists
  3. if parent is owned and snapshot
     • release(parent)
  4. for each owned and snapshot dependency
     • release(dependency)
  5. update parent version (MUST exist now)
  6. update owned and snapshot dependencies (MUST exist now)
  7. set project release version (ie from 1.0.0-SNAPSHOT to 1.0.0)
  8. commit changes to scm
  9. tag freeze on scm
  10. perform all maven lifecycle phases till deploy
  11. set project next version (ie from 1.0.0 to 1.0.1-SNAPSHOT)
  12. commit changes to scm again

Answer 2

You can switch to Gradle, but it is probably easier to continue to use Maven with a few minor modifications:

• Add <modules> in your parent pom in which you specify all your child modules.
• Set the parent version in your child poms to be the same as your parent version, e.g. 1.0.0-SNAPSHOT.
• Add the project that gets included by the other builds, i.e. bbb and ccc to the <dependencyManagement> section of your parent pom, set their version to be ${project.version}, and remove their specific versions in ddd and eee.
• Likewise, add the aaa plugin to the <pluginsManagement> section of your parent pom, set its version to be ${project.version} and remove the specific version from the other projects.

Now you can build and release all your builds in one go, e.g. by using the maven-release-plugin.

Answer 3

The guys from barchart created a Jenkins plugin: "Maven Cascade Release Plugin"

https://github.com/barchart/barchart-jenkins-cascade-plugin/wiki/User-Manual

This requires that you create a separate 'Layout' project that manages the order in which other projects are to be released:

Root layout:
<project>
  <modules>
    <module>a</module>
    <module>ant</module>
    <module>fish</module>
    <module>fish/salmon</module>
    <module>fish/shark</module>
</modules>

This project still needs to be in Subversion, it seems, but you can put it in a separate repository, and link the actual projects with svn:externals for example. The plugin in Jenkins will then figure out to release one module which modules it needs to release first.

E.g to release fish/shark -> 1.0, it could need to release:

• a -> 1.1
• fish/salmon -> 1.2
• fish/shark -> 1.0

Answer 4

You haven't mentioned that in your parent POM you are using "modules" tag. I propose you should use it - that way you can specify the line up for the project's build process.

To solve your current problem, create seperate module/project. In pom specify profiles to trigger ant operations. This should do the trick.

You would call the mvn command on parent dir as follows, i.e.:

mvn clean install -Pdeploy

Answer 5

Change the structure in SVN according to your dependencies:

In parent you have to define the list of modules and the dependencies between the modules have to be defined in the appropriate module.

parent (pom.xml)
 +-- aaa (plugin)
 +-- bbb (jar)
 +-- ccc (jar)
 +-- ddd (war)
 +-- eee (war)

The version should be the same for all modules like 3.0-SNAPSHOT after that you can simple release/deploy etc. from the root location.

If this is not the way you would go you should go via Jenkins to release all these projects separately.

Answer 6

Now it's possible your modules are too fine grained, but personally I think the separate builds is the way to build you system and here's why.

The parent pom's primary job is to represent the next release. That release could be a feature release involving several in-house modules or it could be a bug-fix release in which only a single module changes.

Given this the cycle goes something like this:

1. modify the parent pom's version to correspond to the next 'release'.
2. update the module versions being modified as part of the release.
3. update the modules to use the snapshot modified parent and make the changes.
4. update the module versions in the parent to their release version and release the parent.
5. update each module to use the released parent pom and release the modules.

The benefit of this approach is that, where a module is not modified you use the module version that were previously released (obtained from your release repo, so that's not 100% guaranteed) without a rebuild (which would happen with a multi-module approach) reducing the regression testing overhead.

If the number of modules changing with each release becomes a burden, it may be time to review your module structure, however the benefit of this approach is that the parent pom aligns all module versions across your system allowing reducing the regression testing effort and making it easy to make quick small released.