automate deployment to sonatype's oss maven repository

Question

I've got several github java projects. One of them I've manually deployed to sonatype's repository so that it gets published in maven central.

This has been a somewhat painful process in the sense that it seems to involve too many hoops to jump through and a lot of manual work and I'd like to automate that. So I actually stopped doing that because it was just too much work. There's plenty of documentation that suggests this is possible and quite a bit that suggest that it somehow involves doing something with the nexus-staging-maven-plugin. Unfortunately all of that documentation is (in typical maven style) skipping over the essential details that would allow me to figure out in a straightforward way the minimum amount of steps necessary that allow me to automatically publish release builds to the sonatype repository (i.e. without me manually approving things).

So, what is the blurb that needs to be present in my pom (assume a otherwise bog standard uncomplicated java project), including urls for the sonatype repository, all documentation I've found seems to insist localhost:8081 is it, and the required maven incantations to make it do a release (preferably via the mvn release plugin), have it sign the artifacts, and have it deploy the resulting artifacts to sonatype, approved and all ready to be synced to maven central, etc.

So, I'm sort of looking for the maven replacement of a "gem push" in the ruby world, which gets the job done in a convenient one liner. It's a simple case of given a jar file approved by me, how do I get it to end up in maven central with the least amount of fuss.

I'd very much appreciate some examples of pom files already setup to do this that I can copy and adapt.

Edit:

Here's my working pom file:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.jillesvangurp</groupId>
    <artifactId>jsonj</artifactId>
    <version>1.34-SNAPSHOT</version>

    <name>JsonJ</name>
    <description>A framework for working with json in Java the "proper" way. No mappings or model classes, it's all just lovely json, but in Java.</description>
    <url>https://github.com/jillesvangurp/jsonj</url>

    <licenses>
        <license>
            <name>MIT license</name>
            <url>https://github.com/jillesvangurp/jsonj/blob/master/LICENSE</url>
            <distribution>repo</distribution>
        </license>
    </licenses>

    <scm>
        <url>git://git@github.com:jillesvangurp/jsonj.git</url>
        <connection>scm:git:git@github.com:jillesvangurp/jsonj.git</connection>
        <developerConnection>scm:git:git@github.com:jillesvangurp/jsonj.git</developerConnection>
    </scm>

    <repositories>
        <repository>
            <id>sonatype-nexus-snapshots</id>
            <name>Sonatype Nexus Snapshots</name>
            <url>https://oss.sonatype.org/content/repositories/snapshots</url>
            <releases>
                <enabled>false</enabled>
            </releases>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
        </repository>
    </repositories>

    <distributionManagement>
        <snapshotRepository>
            <id>sonatype-nexus-snapshots</id>
            <name>Sonatype Nexus Snapshots</name>
            <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
        </snapshotRepository>
        <repository>
            <id>sonatype-nexus-staging</id>
            <name>Nexus Release Repository</name>
            <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
        </repository>
    </distributionManagement>

    <developers>
        <developer>
            <id>jillesvangurp</id>
            <name>Jilles van Gurp</name>
            <url>http://www.jillesvangurp.com</url>
            <timezone>gmt+1</timezone>
            <roles>
                <role>Main Developer</role>
            </roles>
        </developer>
    </developers>

    <organization>
        <name>www.jillesvangurp.com</name>
        <url>http://jillesvangurp.com</url>
    </organization>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.5.1</version>
                <configuration>
                    <verbose>true</verbose>
                    <fork>true</fork>
                    <source>1.7</source>
                    <target>1.7</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-javadoc-plugin</artifactId>
                <version>2.8.1</version>
                <executions>
                    <execution>
                        <id>documentation</id>
                        <phase>prepare-package</phase>
                        <goals>
                            <goal>jar</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-source-plugin</artifactId>
                <version>2.2</version>
                <executions>
                    <execution>
                        <id>gathersource</id>
                        <phase>prepare-package</phase>
                        <goals>
                            <goal>jar</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.sonatype.plugins</groupId>
                <artifactId>nexus-staging-maven-plugin</artifactId>
                <version>1.6</version>
                <extensions>true</extensions>                
                <configuration>
                    <!-- The Base URL of Nexus instance where we want to stage -->
                    <nexusUrl>https://oss.sonatype.org/</nexusUrl>
                    <serverId>sonatype-nexus-staging</serverId>
                </configuration>
            </plugin>            
        </plugins>
        <extensions>
            <extension>
            <artifactId>wagon-webdav-jackrabbit</artifactId>
            <groupId>org.apache.maven.wagon</groupId>
            <version>2.2</version>
            </extension>
        </extensions>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-release-plugin</artifactId>
                    <version>2.1</version>
                    <configuration>
                        <mavenExecutorId>forked-path</mavenExecutorId>
                        <useReleaseProfile>false</useReleaseProfile>
                        <arguments>-Psonatype-oss-release</arguments>
                    </configuration>
                </plugin>
            </plugins>
        </pluginManagement>        
    </build>
    <profiles>
        <profile>
            <id>sonatype-oss-release</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-source-plugin</artifactId>
                    </plugin>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-javadoc-plugin</artifactId>
                    </plugin>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-gpg-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>sign-artifacts</id>
                                <phase>verify</phase>
                                <goals>
                                    <goal>sign</goal>
                                </goals>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>        
    </profiles>

    <dependencies>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>6.8.7</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>junit</groupId>
                    <artifactId>junit</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.hamcrest</groupId>
            <artifactId>hamcrest-all</artifactId>
            <version>1.3</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.googlecode.json-simple</groupId>
            <artifactId>json-simple</artifactId>
            <version>1.1.1</version>
            <exclusions>
                <exclusion>
                    <artifactId>junit</artifactId>
                    <groupId>junit</groupId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>xom</groupId>
            <artifactId>xom</artifactId>
            <version>1.2.5</version>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>commons-lang</groupId>
            <artifactId>commons-lang</artifactId>
            <version>2.6</version>
        </dependency>
        <dependency>
            <groupId>com.jillesvangurp</groupId>
            <artifactId>efficientstring</artifactId>
            <version>1.11</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-core</artifactId>
            <version>2.2.3</version>
        </dependency>
    </dependencies>
</project>

The comment (@aurelien-thieriot) below put me on the right track but was not enough by itself.

In the end I took the sonatype parent pom and flattened it into my pom file.

This allows me to use the mvn release plugin normally. It uploads the artifacts to the sonatype staging repository. Then to release the artifacts, I actually needed the staging repository id. You can find this from the repositories view in https://oss.sonatype.org/index.html#stagingRepositories.

In my case the command line became:

mvn nexus-staging:release -Ddescription="Release 1.33" -DstagingRepositoryId=comjillesvangurp-1002

Without the right id it doesn't figure it out and still fails: Sonatype Maven Staging Plugin Issue

So 95% automated but I still need to figure out the stagingRepositoryId every time.

Edit:

mvn release:perform actually tells you the id of the staging repository. I guess you could write a script that extracts this id from the output and then passes it in to the next step. If somebody knows some mvn voodoo to make mvn release:perform do the staging release as well, it would be much appreciated.

Answer 1

For the convenience of Maven projects, Sonatype is providing a parent POM you can add to your project with all the basic configuration:

https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide#SonatypeOSSMavenRepositoryUsageGuide-Changesto%7B%7Bpom.xml%7D%7D

The important bits are:

  <parent>
    <groupId>org.sonatype.oss</groupId>
    <artifactId>oss-parent</artifactId>
    <version>7</version>
  </parent>

And the source code repository details:

  <scm>
    <connection>scm:svn:http://foo.googlecode.com/svn/trunk/</connection>
    <developerConnection>scm:svn:https://foo.googlecode.com/svn/trunk/</developerConnection>
    <url>http://foo.googlecode.com/svn/trunk/</url>
  </scm>

You will also need GPG to be install on your computer (Required to sign the packages) and our settings.xml correctly filled with your credentials:

  <servers>
    <server>
      <id>sonatype-nexus-snapshots</id>
      <username>your-jira-id</username>
      <password>your-jira-pwd</password>
    </server>
    <server>
      <id>sonatype-nexus-staging</id>
      <username>your-jira-id</username>
      <password>your-jira-pwd</password>
    </server>
  </servers>

After that, you should be able to use the two steps release:

$ mvn release:prepare

$ mvn release:perform

Unfortunately, I don't know any way of automate the manual approval part of the process (In oss.sonatype.org). But that should already save you some times.

The documentation, as shown above, is probably a bit convoluted but is very complete and gives you all you need to know for various scenarios.

EDIT:

In fact I think I am wrong and there is a part on automate approval process. Interesting.

And for this part you are right, the details are quite limited. Though, I hope the first part of the configuration already helps you a little bit. I need to look further into this staging stuff (Or maybe someone else would have already done it !)

EDIT_AGAIN:

I need to actually try it but it would sound like something as follow:

        <plugins>
            <plugin>
                <groupId>org.sonatype.plugins</groupId>
                <artifactId>nexus-staging-maven-plugin</artifactId>
                <version>1.6</version>
                <extensions>true</extensions>
                <configuration>
                    <!-- The Base URL of Nexus instance where we want to stage -->
                    <nexusUrl>https://oss.sonatype.org/service/local/staging/deploy/maven2/</nexusUrl>
                    <serverId>sonatype-nexus-staging</serverId>
                </configuration>
            </plugin>
        </plugins>

According to the documentation, the deploy should be replaced by the right staging workflow (Including the close) and it would left the latest step:

$ mvn nexus-staging:release -Ddescription="Yippie!"

TO BE TESTED...

Answer 2

So 95% automated but I still need to figure out the stagingRepositoryId every time.

You can use mvn nexus-staging:rc-list

Specifically, by doing mvn release:rc-list and using grep or whatever to filter the output from that by some form of the group ID or other substring that you know the stagingRepositoryId to be, you can determine the full stagingRepositoryId value

For example, the group ID for my project is nu.validator and my stagingRepositoryId values are all in the form nuvalidator-NNNN where the NNNN part is a number that started from 1000 with my first release and that the system increments by 1 each time I release; so nuvalidator-1000, nuvalidator-1001, and so on.

So in the python script I use for my build, I just do this:

output = subprocess.check_output("mvn nexus-staging:rc-list  -DnexusUrl=https://oss.sonatype.org/ -DserverId=ossrh")
for line in output.split('\n'):
    if "nuvalidator" in line:
        stagingRepositoryId = "nuvalidator-" + line[8:23]
        ...

That's because the relevant lines returned in the mvn nexus-staging:rc-list output are in the form:

...
[INFO] central_bundles-3514 OPEN     Implicitly created (auto staging).
[INFO] central_bundles-3515 OPEN     Implicitly created (auto staging).
[INFO] central_bundles-3521 OPEN     Implicitly created (auto staging).
[INFO] nuvalidator-1008     OPEN     Implicitly created (auto staging).
...