Managing Maven dependencies between multiple components with multiple teams making changes

Question

In a situation where there are multiple teams making changes to some Maven projects with dependencies between them (otherwise unrelated projects i.e. no super POM or anything like that), with each team working in a separate working branch on all of these projects and merging them back to trunk only after a story is complete and tested, how should one handle the dependencies between snapshot builds that each team produces?

Note that any of the teams could be making changes in any of the projects: they work on different pieces of functionality of the same system, and the projects are more like different layers.

Before the split into multiple teams, artifacts were deployed to a central Nexus repository. Of course, it would be possible to set up separate Nexus repositories for each team, and an additional one for trunk, but I'm hoping there would be a better way to deal with this.

Answer 1

Source code repositories have branches, binary ones don't. So any part of your development plan that relies on branching can't rely on binary repositores for things that are in the scope of branches.

(One exception to the above is ClearCase, which does do both, and is best not spoken of.)

If the teams are to be independant, you really do need to manage the dependencies in terms of released versions. This will be painful; what you are doing is hard. Probably have to get used to planning and discussing changes in advance, abandoning ad-hoc refactoring.

On the other hand, perhaps the teams are really only nominal, e.g. everyone is all working together to a single overall product release/delivery schedule. Then just keep to a single overall version number that gets bumped any time a feature branch is merged back to master. All local dependencies then simply use {project.version}.

Note that this is still not a snapshot, as you never update it without changing the number.

And then within a team share code not binaries. Some extra recompilation, but if that takes more than 15 seconds, get faster PCs. And anyone not using a multi-module IDE like eclipse will have to write a trivial reactor-build POM to cover the set of modules they are currently working on.

See:

http://books.sonatype.com/m2eclipse-book/reference/eclipse.html

http://axelfontaine.com/blog/final-nail.html

http://ahoehma.wordpress.com/2010/12/22/intermodule-dependencies-now-better-working-with-maven-3/

For how to set up an IDE, CI build server and reactor build for this kind of situation.

Or, you could try reimplementing ClearCase with some scripting on top of git+Nexus (don't do this).

Answer 2

TL;DR: Use release dependencies instead of snapshot ones.

---

Case to consider when choosing between snapshot and release dependencies is when changes in some component (A) break a dependent component (B).

When this happens, you need to fix it, but there are two options, one of which might turn out preferable depending on your project specifics.

One option is when you prefer to fix the issue immediately, even though it may come at cost of distracting developer working on component A. Benefit of this approach is its conceptual simplicity - it relieves developers from the need to memorize, maintain and update dependencies between components, because instead their efforts are focused on simply keeping all the most recent versions of components in sync.

In this case, release dependencies don't offer anything appealing compared to snapshot ones; rather the opposite - these would introduce an extra step / effort to release updated component A to be able to test whether dependent component doesn't break anymore.

• ^{By the way, Maven provides support for a simplest form of snapshot dependency preference: multi-module projects, where handling snapshot dependencies between modules is made as easy as it gets (though this comes at the expense of simultaneous releases, which is sometimes undesirable - eg when components / modules tend to change at very different pace).}

An opposite option is when you would prefer to postpone integration fixes in the component A, for example when you believe that in your project it is more important for developer to strongly focus on the changes made in it and switch their efforts to integration later. This could be the case when dependencies between components are "loose", small, carefully designed and slowly changing interfaces.

• ^{For example, if you already have carefully crafted, thoroughly tested, clean and convenient interface between components A and B, you may prefer to postpone messing in B while you're polishing changes roughly drafted in A, expecting that it will take less effort to integrate after interface at A side gets clean back again.}

In cases like this release dependencies allow you to totally isolate dependent component. You don't need to worry about component B at all, until you are done with A and modify dependency in B to refer desired release. Snapshot dependencies can not offer anything like this: if changes made in A break dependent component, you'll have to put an effort into bringing things back to shape.

---

Now, let's see how above applies in your case,

• unrelated projects
• each team working in a separate working branch
• split into multiple teams

All of the above whispers, says, cries that team is moving in a direction allowing more independent, more isolated work on the components of the system. So that John working on a project A, Paul working on B, George working on C and Ringo working on D have a leeway in planning and implementing changes within these components without permanently having to look at what happens in other components.

Now, let's look from above perspective at "dependencies between snapshot builds that each team produces". The way how Maven works, this would mean a huge step back into what team is supposed to move away from. the very moment John makes an intermittent change in A and builds a snapshot, Maven immediately picks that snapshot and "broadcasts" it into projects of Paul, George and Ringo!

Now, three guys working in other projects have to adapt to changes made by John and, if these changes cause some serious trouble in other projects, most likely John will have to break his flow and urgently adapt their project to... oh by the way to what is he going to adapt? Paul, George and Ringo, they all work like John, their projects are permanently changing, are they supposed to explain to John what exactly he needs to take into account for today, and what he possibly will have to account for tomorrow?

You see, if team is supposed to move to more independent development of different components / projects, using snapshot dependencies essentially totally breaks this movement and returns you back to the days of monolithic approach.

If you want to keep moving to stronger separated components / projects, you better have to define "communication points" at which these projects would "declare" their readiness / obligation to integrate with others. In Maven, releases naturally serve this purpose.

Maven releases provide stable, fixed "checkpoints" allowing project developer to safely communicate with outer world.

• ^{Release 2.15.23 is most recent one ready to be integrated; we no longer support / accept bug reports against releases earlier than 2.14.999; releases 2.15.24 and later ones currently suffer from a critical issue and are by no means intended for integration until further notice.}

Worth stressing that release dependencies denote stable "checkpoints" between components, not between people. Whether components are maintained by separate developers or not isn't really relevant.

Even if you have, say, John working in both components A and B or maybe John and Paul together working on these two components, it still makes sense to have release dependencies, to allow programmer focus on work in currently chosen component, without being permanently distracted with worries about how this could impact another.

Given above, you also need to decide between teams how exactly are you going to schedule releases to make integration as smooth as possible. Here your options are really broad. A natural "schedule" that comes to mind is that each team decides when they're ready to offer a "checkpoint" of their work for other teams to integrate.

However natural ad-hoc releases sound "from within" a single team, if you look at it from the perspective of other teams, you may find that they would prefer to have a more regular way to get to "integration checkpoints" - from this perspective, you may wish to consider some predictable ways to "promote" component updates for other teams attention - monthly, or weekly, or maybe daily releases.

---

Now that your system became more complicated, specifics and purpose of snapshot dependencies are worth taking a closer look at. You can find a pretty decent explanation of this in Maven: The Complete Reference at Sonatype (bold font in below quote is mine):

...when you deploy a snapshot, you are not making a release of a software component; you are releasing a snapshot of a component at a specific time.

Why would you use this? SNAPSHOT versions are used for projects under active development. If your project depends on a software component that is under active development, you can depend on a SNAPSHOT release, and Maven will periodically attempt to download the latest snapshot from a repository when you run a build. Similarly, if the next release of your system is going to have a version "1.4", your project would have a version "1.4-SNAPSHOT" until it was formally released...

If a project depends on a SNAPSHOT, it is not stable as the dependencies may change over time... SNAPSHOT versions are for development only...

As you can see, above stresses temporary, intermittent, unstable nature of snapshot dependencies and limits their applicability to times of "active development". In your case, you can think of temporarily switching to snapshot dependency when you need to do series of closely correlated, synchronized, tightly coupled changes in different components. After this "active" phase is over, you can do "checkpoint" releases of involved components and switch back to release dependencies.

Note by the way that if you discover that switching to snapshot dependency happens too often, this might indicate a need to rethink your design. You may think of ways to better isolate components, or of opposite, that is, of "merging" previously separate components back into monolithic, synchronously developed and released module.