git archive export with submodules (git archive all / recursive)

Question

I have a website directory versioned with git. I use submodules for required libraries like Twitter Bootstrap, colorbox and lessjs because I should not track the sourcecode but only the version of their code I use.

Now I want to deploy the project and therefore I need the submodule code too. By using git archive I don't get the source files/code from the submodules.

The following three approaches TRY to achieve what I want but dont't work: 1st approach

#!/bin/sh

export revision="$1"

export GIT_INDEX_FILE=".git/tmpindex"
rm -f "$GIT_INDEX_FILE"

git read-tree $revision

export up="$(pwd)"

read_one_level () {
        export GIT_ALTERNATE_OBJECT_DIRECTORIES="$GIT_ALTERNATE_OBJECT_DIRECTORIES":$(
            git submodule foreach 'echo "$up/$path/.git/objects"' |
            grep -E -v '^(Entering|No submodule mapping found)' |
            tr '\n' : |
            sed 's/:$//'
        )

        git submodule foreach '
                cd "$up"
                subcommit=$(git rev-parse :"$path")
                git rm --cached "$path"
                git read-tree -i --prefix="$path/" $subcommit
        ' >/dev/null
}

while git ls-files -s | grep -q ^160000; do
    read_one_level
done

git archive --format=tar $(git write-tree)

rm -f "$GIT_INDEX_FILE" 

Thomas Rast in http://git.661346.n2.nabble.com/Running-git-archive-recursively-over-submodules-td4577012.html

This gives me errors both on Windows as in Linux that there are no objects files found.

2nd approach https://github.com/meitar/git-archive-all.sh

Complains about mktemp not found on Windows. And correcting calls to git archive (from git-archive) does not give submodules in the tar ...:(

3rd approach https://github.com/Kentzo/git-archive-all

Is outdated from my point of view by not being compatible to latest python 3.3 and still not fully working by using 2.7 because samefile complains.

So my question is now: Is there any recent way/approach to deal with exporting/archive a git project including submodules?

Or should I check subtrees for this workflow?

Thanks in advance

Answer 1

https://github.com/Kentzo/git-archive-all is now working under linux as @Kentzo stated.

In the cygwin environment (with git and python installed via cygwin installer) it works there too.

Answer 2

I'm using the following code

git archive -o release.zip HEAD
git submodule --quiet foreach 'cd $toplevel; zip -ru release.zip $sm_path'

to create a complete archive of a git repository with all submodules.

If you want to be fancy you can even rewrite the zip comment via

echo -e "Repository:\n$(git rev-parse HEAD)\nSubmodule status:\n$(git submodule status)" | zip -u release.zip -z

All on windows using infozip.

Answer 3

We could do a quick bash script, where archive in tar, module and sub-modules, after we concatenate tars and compress, here it is an example with 2 sub modules:

#!/bin/bash

set -e

pwd=$(pwd)
date=$(date +%Y%m%d)
package=gitreponame
branch=master
name=tarbz2name
submodule1=/src/app/sub1/
submodule2=/src/sub2/

pushd ${package}
git checkout ${branch}
git pull
tag=$(git rev-list HEAD -n 1 | cut -c 1-7)
git archive --prefix="${name}/" --format=tar master > "$pwd"/${name}-${date}-${tag}.tar

git submodule update --init

cd ${submodule1}
git archive  --prefix="${name}${submodule1}" --format=tar master > "$pwd"/sb1.tar
cd -

cd ${submodule2}
git archive --prefix="${name}${submodule2}" --format=tar master > "$pwd"/sb2.tar
popd

tar -Af ${name}-${date}-${tag}.tar sb1.tar
tar -Af ${name}-${date}-${tag}.tar sb2.tar
bzip2 ${name}-${date}-${tag}.tar
rm sb1.tar sb2.tar

Answer 4

The 2nd approach: https://github.com/meitar/git-archive-all.sh deploys as a standalone Bash script and appears to be still-maintained. I found it to be an easy way to take a quick submodule inclusive snapshot of a project. It can even capture an earlier revision (sort of, see below):

git-archive-all.sh --tree-ish (tag) ## for example

A couple of problems that may or may not be deal breakers:

• I ran into problems trying to use the --format option -- but the default, tar, generally worked.

• The (very) big problem: It captured my submodule, but only in its current version -- not the specified version. Whoops!

Being a Bash script, this probably requires some kind of Linux compatibility layer (like Cygwin) to be run under Windows.

---

It seems like this should be a built-in capability of Git -- it's too complicated to piece together with scripting. I guess not many people use submodules or git-archive, so even fewer want to use both.

Answer 5

If you are a friend of KISS as me you could use the answer of @t-b BUT as I found this solution does not archive nested submodules. The following code will help

# archive main directory
$ git archive --format tar -o release.tar HEAD
# descend recursively and archive each submodule
$ git submodule --quiet foreach --recursive 'git archive --format tar --prefix=$displaypath/ -o submodule.tar HEAD'
# concatenate with main archive
$ TOPDIR=$(pwd) git submodule --quiet foreach --recursive 'cd $TOPDIR; tar --concatenate --file=release.tar $displaypath/submodule.tar; rm -fv $displaypath/submodule.tar'
$ gzip -9 release.tar

The result will be file result.tar.gz ... instead of HEAD you could choose another commit, of course.

Answer 6

I have developed git-archive-with-submodules using bash. It also has the nice feature of allowing to create an archive including uncommitted changes.