git smudge/clean filter between branches

Question

There are many related questions involving smudge/clean filters - I have spent some hours reading them, and trying various options, but still failing. I hope I can ask in a way that I get an answer that works for me.

Specifically, I have read the page most of these answers link back to:

• Customizing Git - Git Attributes

---

tl;dr

Its a detailed question, but the summary is:

• Can I store DEBUG = false in a file on one branch, and DEBUG = true in another branch, using smudge/clean filters to manage that file? And how?

---

Background

I have various remote repos hosted at bitbucket. I am using SourceTree on Win8, to clone the remote repos to my laptop. I create different branches for development, features, releases etc (following A successful Git branching model for better or worse).

I have an Android java class called Dbug.java that contains a boolean which turns on/off various debug logging, mocking etc features in my code.

public static final boolean DEBUG = false;

I would like this value to be false on my "production" (master) branch, and to be true on my feature branches.

• Is this possible using filters, or have I already misunderstood the use case?
• I am unsure if filters work like this between 2 branches of the same locally hosted repo, or if the filters only work between 2 repos.

---

Creating the filters

Working locally, I checked out the production branch. I created a test file called debug_flag.txt with the following contents:

// false on production branch
// true on other branches
DEBUG = false;

I created a file in the root of my local repo called .gitattributes and added the filter reference to it:

debug_flag.txt filter=debug_on_off

I updated the .git/config file with the filter definition:

[filter "debug_on_off"]
    clean = sed -e 's/DEBUG = true/DEBUG = false/'
    smudge = sed -s 's/DEBUG = false/DEBUG = true/'

• In my understanding, this should ensure that my file always has a false value in production, but will have a true value when I branch from production.
• Is this a correct understanding?

---

Testing the filters

I created a new branch test using:

git checkout -b test

I checked the contents of my file:

$ cat debug_flag.txt

// false on production branch
// true on other branches
DEBUG = false;

• I expected to see the value true in the file
• Shouldn't the "smudge" filter have run when I checked out the file?

I added a new line to the file, and committed. I then switched back to the production branch, and this is where things get weird.

If I look at the file in SourceTree, there are no changes on this branch since it was created. That is what I would expect, since the only change was made on a different branch.

If I look at the file in the terminal, or Notepad++, I see my value has changed:

$ cat debug_flag.txt

// false on production branch
// true on other branches
DEBUG = true;

I have not yet merged the change across from the test branch, I have not made a commit on the production branch, yet the file has changed.

• it looks like the smudge filter was run on the file within this branch, but not across branches.

---

I'm missing a vital piece of the puzzle, and hopefully it is something simple that can be spotted by someone with experience doing this.

My bet is this is a simple misunderstanding of the concept.

Pls prompt for any missing info...

---

Update based on VonC's reply

Setting up the basic filters worked quite well. Defined the filter in the config file as:

[filter "debug_on_off"]
    clean = sed -e 's/DEBUG = true/DEBUG = false/'
    smudge = sed -s 's/DEBUG = false/DEBUG = true/'

Creating a new branch fixes false -> true, merging back changes true -> false.

Confining the change to just the production (master) branch required custom scripts that were aware of the branch they are being run from. So the config file became:

[filter "debug_on_off"]
    clean = ./scripts/master_clean.sh
    smudge = ./scripts/master_smudge.sh

master_clean.sh:

#!/bin/sh
branch=$(git rev-parse --symbolic --abbrev-ref HEAD)
if [ "master" = "$branch" ]; then
    sed -e s/DEBUG = true/DEBUG = false/ $1
else
    cat $1
fi

master_smudge.sh:

#!/bin/sh
branch=$(git rev-parse --symbolic --abbrev-ref HEAD)
if [ "master" = "$branch" ]; then
    sed -e s/DEBUG = false/DEBUG = true/ $1
else
    cat $1
fi

At this point, I am running into inconsistencies between what SourceTree is seeing, and what is being shown in Notepad++ for the contents of the debug file. SourceTree is showing the changes, but Notepad++ is not.

I am accepting VonC's answer, since it answers the basic question I posed.

However, I will likely be implementing the solution I wrote, since it solves the underlying problem that I am trying to solve, in an easier way (for me): retaining a different config file on separate branches.

Answer 1

I expected to see the value true in the file

You just created a new branch, not checked out its content (sice its content is the same as the branch you were in)

To force the smudge to run, do at the top of the repo:

git checkout HEAD --

I have not yet merged the change across from the test branch, I have not made a commit on the production branch, yet the file has changed.

That is the idea of a content filter driver: it modifies the content, without affecting git status (which still reports the modified file as "unchanged").

To have a smudge acting differently per branch, I would recommend calling a script which starts by looking the name of the current branch.
See an example in my older answer "Best practice - Git + Build automation - Keeping configs separate".

#!/bin/sh
branch=$(git rev-parse --symbolic --abbrev-ref HEAD)

Answer 2

VonC's advice addresses the exact question I posed, but I was unable to work out the final details (as per my Update to the question). This answer gives the details of how I have done things.

---

Update

Below method worked for the first merge. But after that it is no longer working. I'm leaving it here, since it represents the current state of my investigation.

It seems that the merge drivers are no longer being called.

Also tried various modifications from related questions using exit 0, touch %A, or a custom script merge driver (https://stackoverflow.com/a/930495/383414) instead of true as presented below.

---

I found a workaround to this that uses a custom merge strategy to solve the underlying problem, which is:

• I want to have build files in my build branch always set to have all debug values turned off.
• This prevents any accidental releases of the product with mock settings, localhost settings, logging turned on etc.

I have based the following on info from this question: .gitattributes & individual merge strategy for a file

1) Define a custom merge driver in the .git/config file as follows:

[merge "ours"]
    name = "Keep ours merge"
    driver = true

I am not sure if this step is required - but it seems it may be a workaround for a bug on some (older?) systems.

(for details: https://stackoverflow.com/a/13000959/383414)

2) Set up a .gitattributes file in the build/production/pristine branch so that the special debug flag uses the above merge strategy.

So using the files in my question, go to the "production" branch, and add the following line to the .gitattributes file:

debug_flag.txt merge=ours

Any time a merge is made back to the "production" branch, git will look for the merge strategy defined as "ours", and will prevent debug_flag.txt from being overwritten.

3) On the other branches, set up your .gitattributes file without that custom merge strategy.

4) Last (but important) step of the config process, is to set up the debug_flag.txt file correctly in all branches, and to commit changes to each branch.

You should now have 2 branches, each containing different versions of .gitattributes & debug_flag.txt files. This ensures that on each time you merge, there are conflicts.

Without conflicts, the custom "ours" merge strategy is not called, and the files could get overwritten.

(for details: https://stackoverflow.com/a/16739788/383414)

5) Finally merge your new branch back into "production". You will have merge conflicts due to steps 3 & 4. Resolve the conflicts so that the 2 branches keep their differences. Commit the changes.

All future merges between these 2 branches will ignore the debug_flag.txt file seamlessly.

---

This accomplishes the goal of having 2 different config files on different branches, so that you can easily separate debug from production code etc. It seems to be a common use case, with many related questions on this forum, but it still took me a couple of days to get it right.

Answer 3

Take a look at expandr. It's a script that lets you set up different configs on different branches using smudge/clean. Basically just what you were originally asking for.

Right now the biggest gotcha is just that after switching branches sometimes I need to do a git checkout HEAD -- "$(git rev-parse --show-toplevel)" to get the working directory to be correctly smudged. Other times, however, it seems to work fine; I haven't yet figured out why. I think it might have to do with "merge renormalize" being turned on, causing some problems? I'm not sure. (I have it on.)

The other gotcha is that you must protect each branch's .gitattributes files themselves with merge=ours by putting a line in that says .gitattributes merge=ours and of course turning the driver on for that (as you already mentioned). The gotcha here is that after you create each separate branch, now you must go into each .gitattributes file and modify each one (I recommend now adding a comment like #touched for merge=ours on master, #touched for merge=ours on test branch, etc., so you'll remember why it was there). You must do this because merge=ours will only protect a file from changing into the version of the incoming branch in a merge if that file has been changed after branch creation on both the incoming branch and its parent branch. (Remember git deals with changes, not with files.)

Answer 4

The most straight-forward solution would be to update your makefile to include a check on which branch is currently checked out. If the branch is part of a named list, define a new build argument -DDEBUG=true or -DDEBUG=false otherwise.

See How to programmatically determine the current checked out Git branch

branch_name=$(git symbolic-ref -q HEAD)
branch_name=${branch_name##refs/heads/}
branch_name=${branch_name:-HEAD}

PROD_BRANCHES := master \
                QA
debug_flag=
ifneq ($(filter $(branch_name),$(PROD_BRANCHES)),)
    debug_flag="-DDEBUG=true"
endif
ifeq($debug_flag,)
    debug_flag="-DDEBUG=false"
endif

Answer 5

I have a similar use case and was able to resolve it with a filter and a post-checkout hook. The hook is nice because it does the clean+smudge for you and immediately updates the file when switching branches via GitHub Desktop. With this approach, each sandbox/tier has their own configuration per branch.

Detailed write-up at: https://c1.eagen.net/git-smudge-clean.html

Sample repository at: https://github.com/vinnyjames/git-filter-demo.git