Is there a faster way to initiate a new repo?

Question

I have a problem with initiating a new git repo in an existing directory. We are currently receiving a tar file with a file system that we unpack, create a git repo from and then want to run tests on. The reason why the file system needs to be a git repo is because our test flow is dependent of this.

What we are doing right now is this:

tar xf <file.tgz>
git init
git add .
git commit

The problem we are having with this is that "git add ." takes about 10 mins at the moment. Is there any way to do this faster? Like an option to "git init" that adds all files in the directory when the repo is initiated? Doesn't seem lie it in the git manual, but I thought I ask the question just in case someone has any input on it.

The alternative to this is if there's a way to fetch changes in a local repo that's not yet been committed. The files already exists in a git repo from the beginning, but for reasons I'm unsure of we don't want to commit the changes in this repo before it's gone through the delivery checks.

Best regards, Johan

Answer 1

There is no faster way to do this. git add will create Git objects for every file you add, so if your working directory is large, it can take a while until Git has done that.

Usually, chances are high though that not every file changed since the last run, so keeping the repository where it is, and just replacing every file might be able to save you some time as Git won’t create new objects for files it already knows.

However, if you already have a repository anyway, then all that makes little sense. You should set up a correct continuous integration setup to reuse the existing repository structure. Commits happen locally, to your local repository. So just because you commit something, that does not mean that it is shared with others. So if you set up your test server with a separate repository, you could push to the test repository, trigger the tests and then fix all issues before finally pushing it to your central repository where everyone else has access.

Answer 2

The correct solution

Why not commit the changes? Git is a distributed system, so you might commit your changes on a throw-away branch or on a "stable" branch, then tag the commit and rewind the branch to where it was at before the commit—thus having the branches undisturbed and the commit tagged. You might even go for detached HEAD and record the new commit there, then push it to the repo in which the tests are to be run.

I would recommend you to research these possibilities and convince your management this is the way to go: there's no need to touch the branch on which the delivered product is maintained to do testing.

If the machines with these repositories are not easily connected, and thus why you need to use tarballs, consider using git bundle which has been implemented precisely for the means of transferring Git commits over the sneakernet.

With it, your workflow would be like this:

1. In the main repo (say, we're on the "master" branch):

   git checkout -b testing
   ...
   git commit ...
   ...
   git bundle create /tmp/testing.bundle master..testing
   

2. Transfer /tmp/testing.bundle to the target machine.

3. In the target repo:

   git pull /tmp/testing.bundle testing
   

   Or whatever is better fit, like, say

   git fetch /tmp/testing.bundle +testing:testing
   git checkout testing
   

The fallback solutuion

Anyway, if you have to live with this situation, do what @poke suggested: maintain the repository at the testing location, and when you receive another tarball, unroll it over the work tree of that repository and then run

git add --all

To cite the manual on --add:

-A
--all

Like -u, but match against files in the working tree in addition to the index. That means that it will find new files as well as staging modified content and removing files that are no longer in the working tree.

Hence basically it will only add to the index files which were modified.

One potential problem though is deletions: the files which are missing in the tarball compared to the work tree won't somehow be deleted automatically.