git splitting repository by subfolder and retain all old branches

Question 1

Short answer

git filter-branch offers exactly the functionality you want. With the --subdirectory-filter option you can create a new set of commits where the contents of subDirectory are at the root of the directory.

git filter-branch --prune-empty --subdirectory-filter subDirectory -- --branches

Walkthrough

The following is an example to perform this in a safe way. You need to perform this for each subdirectory that will be isolated into its own repo, in this case dir1.

First clone your repository to keep the changes isolated:

git clone yourRemote dir1Clone
cd dir1Clone

To prepare the cloned repository we will recreate all remote branches as local ones. We skip the one starting with * since that is the current branch, which in this case would read (no branch) since we are in a headless state:

# move to a headless state
# in order to delete all branches without issues
git checkout --detach

# delete all branches
git branch | grep --invert-match "*" | xargs git branch -D

To recreate all remote branches locally we go through the results of git branch --remotes. We skip the ones containing -> since those are not branches:

# get all local branches for remote
git branch --remotes --no-color | grep --invert-match "\->" | while read remote; do
    git checkout --track "$remote"
done

# remove remote and remote branches
git remote remove origin

Finally run the filter-branch command. This will create new commits with all the commits that touch the dir1 subdirectory. All branches that also touch this subdirectory will get updated. The output will list all the references that where not updated, which is the case for branches that do not touch dir1 at all.

# Isolate dir1 and recreate branches
# --prune-empty removes all commits that do not modify dir1
# -- --all updates all existing references, which is all existing branches
git filter-branch --prune-empty --subdirectory-filter dir1 -- --all

After this you will have a new set of commits that have dir1 at the root of the repository. Just add your remote to push the new commits, or use these as a new repository altogether.

As an additional last step if you care about the repository size:

Even if all branches where updated your repository will still have all the objects of the original repository, tho only reachable through the ref-logs. If you want to drop these read how to garbage collect commits

Some additional resources:

Question 2

This script does the job for me:

#!/bin/bash

set -e

if [ -z "$3" ]; then
        echo "usage: $0 /full/path/to/repository path/to/splitfolder/from/repository/root new_origin"
        exit
fi

repoDir=$1
folder=$2
newOrigin=$3

cd $repoDir

git checkout --detach
git branch | grep --invert-match "*" | xargs git branch -D

for remote in `git branch --remotes | grep --invert-match "\->"`
do
        git checkout --track $remote
        git add -vA *
        git commit -vam "Changes from $remote" || true
done

git remote remove origin
git filter-branch --prune-empty --subdirectory-filter $folder -- --all

#prune old objects
rm -rf .git/refs/original/*
git reflog expire --all --expire-unreachable=0
git repack -A -d
git prune

#upload to new remote
git remote add origin $newOrigin
git push origin master

for branch in `git branch | grep -v '\*'`
do
        git push origin $branch
done