configuring jenkins to publish open source projects to private nexus

Question

In short, how can i configure jenkins to publish open source artifacts that are NOT being published, to a private nexus repository continuously & automatically, whenever there's a change?

Longer version:

Consider an open source project my code depends on, only the project does not publish itself to any repository. So, if there is no frequent changes in the code, i can clone and publish to my private nexus manually. problem is, the open-source project changes too often, and i don't want to manually track the project state. I want to somehow create a job in jenkins that builds & publishes the open source project to my private repository. The project is a scala + sbt project, and so is my code, so i want to publish ivy style and not, maven style.

my thoughts on this:

Ideally, i would have jenkins run the actions: compile test publish-local, and if all goes well, i'll (somehow) publish the generated artifact. but:

• I don't know how (and if it's possible) to use sbt (or even Ivy) the same way i would use maven to deploy the generated jar.
• I could use a forked project, with a modified build.sbt that will include publishTo key, and a refrence to my .credentials and get jenkins to compile test publish it to my private nexus, but i would have to git pull the changes from the original repo to my forked one, and i'd like this to happen automatically...
• I could use a hacky solution like 2 different jobs jenkins would run. the 1st would be triggered by a SCM pull from the original and would execute a script that would pull the changes from the origin to a forked repo (same as above bullet). the second job, would depend on the first, and would be triggered every time the first job is executed. this job would build and publish the forked repo. but this solution sounds too complicated and hacky. surely there's an easier way to do that...

So, what would be a good approach to deal with a situation like this?

Answer 1

I see two things here:

• You're looking at the SNAPSHOT version of the project and the code is constantly changing. Both Ivy and Maven can be configured to examine the latest SNAPSHOT, and download it if it has changed.
• You're looking at the Release repo, but the official releases are coming fast and furious. You simply want to use the latest. Both Ivy and Maven can be configured to automatically pull the latest available release (although both my complain about the fact you don't have a specified release).

If you use Nexus, you can setup both Ivy and Maven to override the default master repo used, http://repo1.maven.org, and consider your local Nexus repo as the world master of all Maven repos. If your local Nexus repo has that artifact, it'll download that local artifact to the build. If not, your local Nexus repo will go out and fetch the requested artifact.

If you set up your local Nexus repo that way, and you deploy (Maven) or publish (Ivy) your jars to the public repo, the jars will be stored in your local Nexus repository and not the remote repository.

Now, if you want to see if a new version is available, you could use the http://search.maven.org rest API and a bit of scripting elbow grease to check to see if there's a new version of that artifact. Then, you'd setup that repo check as a Jenkins job. Jenkins will run this script, download the latest jar, if it was changed, and then publish or deploy it into your local Nexus repo.

If you want to build the latest, you could setup Jenkins to poll that remote third party repo. If that repo is a Git or Subversion repo, the polling isn't going to take a lot of resources (although I'd only poll that repo no more than once per day just to be kind). Then, that remote repo is merely another local Jenkins job. Add a Publish/Deploy plugins and publish/deploy that artifact to your local Nexus repo.