What is a good versioning strategy to achieve 'fail fast' with chains of dependent maven submodules?

Question

My organisation modularised their monolith into maven modules in dependencies up to four deep. Originally we used the maven plugin to auto-increment the version numbers in the pom every time someone did a checkin. Then parent dependencies could pull in the updates by updating the version numbers in their pom.xml.

This lead to a 'fail slow' problem. Imagine the following dependency scenario - where each letter is a module, and each number is a version:

Now imagine we make a series of changes to D and B and A, forgetting C.

C has gotten left behind (and is still pointing to an old version of D).

Now when the person who has to make a change to C does it - they find that all the changes to D have broken C in multiple ways. Fixing C now takes days.

So the team came up with a different plan. Fail fast.

Now everything that was a version number above is now LATEST. (Note this is maven 2 not maven 3).

This means that any breaking change to D, will show up in B or C the next time it is built. (The key phrase here being the next time it is built).

Now let's assume that module A is the parent web application that developers use. Now a developer checks in a change to B - for which B compiles (and A works on their machine). A however has had several checkins - and the version of B:LATEST now breaks with A:LATEST.

We have a fail-fast strategy - but there is nothing in version control to show what broke the app. (And potentially 40 modules to look at to see what the root cause is.)

The challenge is that you can now have 50 developers whose build is broken, who didn't make a change on their machine - nor see anything in version control. All because their machine pulled in latest from artifactory.

We've had several suggestions to make this better:

1. In the CI tool - tie changes in the module to trigger rebuilds of the parent web app
2. In the CI tool - make module builds email the developers so they know what is going on
3. In the CI tool - make the module build write a 'build message' to a log file in the git repository of the parent web app.

The latest recommendation has been to change the parent web app to version numbers and leave the grandchild modules to LATEST.

Note that for releases - we increment the module versions back to fixed versions.

My question is: What is a good versioning strategy to achieve 'fail fast' with chains of dependent maven submodules?

Answer 1

We have a fail-fast strategy

It sounds like you've already figured out a version strategy that fails fast.

but there is nothing in version control to show what broke the app.

You should have automated integration tests that test the integration between components. If they're failing it should give you a fairly good indication of which component has changed. If it's compilation that's failing, it should be fairly obvious which component is the culprit with a MethodNotFoundException or ClassNotFoundException and the package of the component will be in the stack trace.

The challenge is that you can now have 50 developers whose build is broken, who didn't make a change on their machine - nor see anything in version control. All because their machine pulled in latest from artifactory.

There may actually be something you can do with versions to help resolve this impediment for your team. Using SNAPSHOT versions can actually help you here. Instead of LATEST you could use LATEST-SNAPSHOT or something similar. Using SNAPSHOT is beneficial because there are features in Maven related to keeping versions separate. Artifactory will save all versions of snapshot dependencies in a format like LATEST-SNAPSHOT-20090610.041042-5. Each published version will have a unique identifier and the latest build is addressable by LATEST-SNAPSHOT. This means if your build breaks, you can temporarily revert component B to depend on a specific older snapshot of A component so the build is fixed while B and A are debugged to integrate by one or more members of the team.

Not only that, but you can know exactly which version of LATEST-SNAPSHOT is being used with each build. You can trace it to commits in build logs.

Maven will cache -SNAPSHOT versions for a period of time, and developers and your CI must use the -U (--update-snapshots) flag when building (e.g. mvn install -U) to get the latest snapshot. In the case that something breaks, you can warn your developers not to update their snapshots, or have them use the specific working version of the snapshot until the integration bug is fixed.