Using Hudson and build steps with multiple git repositories

StackOverflow https://stackoverflow.com/questions/1808891

Question

I'm trying out Hudson to replace our current Buildbot setup. I installed the git plugin. Our current setup is like:

ssh://server:/repo/test_framework.git
ssh://server:/repo/project_a.git

Now, to build project_a I added a new job with multiple git repositories (the ones above). I wanted Hudson to clone the repositories into different directories under $WORKSPACE, becase test_framework needs that hierarchy. But Hudson seems to merge everything into $WORKSPACE instead. From the console log:

warning: no common commits
...
[workspace] $ git merge-base ce14a4579e87971659e5e0469136713847055a29 96d2b3c27595de243702414c4358366923696d78
[workspace] $ git merge-base ce14a4579e87971659e5e0469136713847055a29 5bb011b3fa288afd5e4392640b32b8bcc982103e
[workspace] $ git merge-base ce14a4579e87971659e5e0469136713847055a29 aa6ade81669883909ba5f5459a205df1bd0df3c0

Can I configure this in Hudson to better fit our project setup? Do I need to create a local dummy git repository with every project as git submodules or something?

Was it helpful?

Solution

Within Hudson you can chain multiple jobs together. You could try creating separate Hudson jobs for test_framework and another for project_a. Hudson creates a separate directory in $WORKSPACE for each job, so now you should have two different directories under $WORKSPACE.

Setup Chaining

In the job configuration of project_a scroll down to Post-build actions and check Build other projects... Enter in test_framework as the project to build.

In the job configuration of test_framework ensure that Poll SCM is unchecked and that Build after other projects is set to project_a.

How it works

What you have now configured is project_a will poll the SCM looking for changes, when changes are found it will pull them from git. Run build steps (if any) and on completion trigger the test_framework job to pull changes from git (if any) and run its build steps.

OTHER TIPS

The problem with the "Build other projects" solution is that if there are changes to test_framework it will not trigger project_a to build. Instead, I recommend abandoning the git plugin and setting up an "Execute shell" build step with the following:

rm -rf ${WORKSPACE}/*

git clone ssh://server:/repo/test_framework.git ${WORKSPACE}/test_framework
cd ${WORKSPACE}/test_framework
git fetch -t ssh://user@server:/repo/test_framework.git +refs/heads/*:refs/remotes/origin/*
git ls-tree HEAD

git clone ssh://server:/repo/project_a.git ${WORKSPACE}/project_a
cd ${WORKSPACE}/project_a
git fetch -t ssh://user@server:/repo/project_a.git +refs/heads/*:refs/remotes/origin/*
git ls-tree HEAD

Next, create hook files "server:/repo/test_framework.git/hooks/post-receive" and "server:/repo/project_a.git/hooks/post-receive" with the following content:

#!/bin/sh
curl http://hudson/job/job_name/build

Now, whenever changes are pushed to either repository, the hook will use Hudson's API to trigger a build.

I realize this question is very old but I ran into the same problem and used this page to flesh out my own solution that seems to be working really well (even though it is a tad convoluted). Most of the credit for this solution should go to Clinton (the only reason I'm bothering to submit this answer is because his answer doesn't seem to address multiple repositories that need to be in the same base directory).

Suppose you have two repositories (A and B).

Steps:

1) Make two projects to pull code from remote repositories A and B. Put any necessary build steps in either repository.

2) Make a third directory without any source control management. Add a build step to this project to execute a shell command similar to this:

ln -s /var/lib/jenkins/jobs/A/workspace A
ln -s /var/lib/jenkins/jobs/B/workspace B

(Your paths may not be the same. Look them up yourself!)

Now you can add any other build steps that depend on A and B being sisters in a directory. Yay symbolic links!

3) Chain the three tasks together. The order of the pull tasks may or may not matter (you know better than I do) but the task without source control should be the last link in the chain.

I ran into the same problem and currently solved it by creating a job for each project and using the Copy Artifact Plugin to allow building the dependent job even if a Git update is done on it's dependencies (this is to avoid building in the middle of an update to the project which we depend upon).

So project_a would copy the latest stable artifacts it needs from test_framework and an update to test framework would trigger a build in project_a. project_a can still be triggered by a change in Git, it just copies again the latest artifacts in test_framework.

The problem you are describing is already filed as bug in the Jenkins bugtracker: https://issues.jenkins-ci.org/browse/JENKINS-8082

We use the "custom workspace" option in the extended project job configuration to checkout our job's repository into a subdirectory of another job.

That other job checks out the main directory with all submodules:

var/lib/jenkins/jobs/
  + main_job
    + workspace (main git checkout with submodules)
      + modules
        + mod1
        + mod2
  + mod1_job (custom workspace set to main_job/workspace/modules/mod1)
    + workspace (empty)
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top