Is it possible for git-merge to ignore line-ending differences?

Question

Is it possible for git merge to ignore line-ending differences?

Maybe I'm asking the wrong question ... but:

I tried uisng config.crlf input but things got a bit messy and out of control, specially when I applied it after the fact.

For one thing, applying this config after the fact doesn't seem to affect files that were committed to the repository before applying this option. Another thing is that suddenly all commits now result in lots of annoying warning messages about CRLF being converted to LF.

To be honest, I don't really care what line-ending is used, I personally prefer the Unix style \n, but whatever. All I care about, is for git merge to be a bit smarter and ignore the differences in line-endings.

Sometimes I have two identical files, but git would mark them as being in conflict (and the conflict is the whole file) simply because they use a different line ending character.

Update:

I found out that git diff accepts a --ignore-space-at-eol option, would it be possible to let git merge use this option as well?

Answer 1

Update 2013:

More recent git versions authorize using merge with strategy recursive and strategy option (-X):

• from "Git Merge and Fixing Mixed Spaces and Tabs with two Branches":

git merge -s recursive -Xignore-space-at-eol

But using "-Xignore-space-change" is also a possibility

• Fab-V mentions below:

  git merge master -s recursive -X renormalize
  
  

---

jakub.g also comments that the strategies work also with cherry-picking:

git cherry-pick abcd123456 --strategy=recursive --strategy-option=renormalize 

This works much better than ignore-all-space.

---

Original answer (May 2009)

The patch for ignoring eol style has been proposed in June 2007, but it only concerns git diff --ignore-space-at-eol, not git merge.

At the time, the question has been askeed:

Should --ignore-space-at-eol be an option to git-merge ?
Merges are where this functionality matters.
What are the semantics of an auto-resolved merge with those options in effect -- are they only used for rename detection, or do we, e.g., not flag conflicts with only whitespace changes ? And if we don't, which version do we accept automatically ?

Julio C Hamano was not exactly enthusiastic:

This certainly is tempting, but I suspect that should be left to later rounds.
I suspect that it would introduce a concept of two different kinds of diffs, one to be mechanically processed (i.e. use in merge with "git-merge-recursive", and apply with "git-am"), and another to be inspected by humans to understand.
It often may be useful to munge the input for the latter case, even though the output from comparing munged input files may not be readily usable for mechanical application.

The general idea, when it comes to git merge, is to rely on the third-party merge tool.

For instance, I have setup DiffMerge to be the tool for Git merge, setting a ruleset which allow that merge tool to ignore eol for certain type of files.

---

Setup on Windows, with MSysGit1.6.3, either for DOS or Git bash session, with DiffMerge or KDiff3:

• set a directory into your PATH (here: c:\HOMEWARE\cmd).
• add in that directory the script merge.sh (wrapper for your favorite merge tool)

merge.sh:

#!/bin/sh

# Passing the following parameters to mergetool:
#  local base remote merge_result

alocal=$1
base=$2
remote=$3
result=$4

if [ -f $base ]
then
    #"C:/Program Files/SourceGear/DiffMerge/DiffMerge.exe" "$alocal" "$base" "$remote" -m --result="$result" --title1="Mine" --title2="Merging to: $result" --title3="Theirs"

    # for merge respecting eol, KDiff3 is better than DiffMerge (which will always convert LF into CRLF)
    # KDiff3 will display eol choices (if Windows: CRLF, if Unix LF)
    "C:/Program Files/KDiff3/kdiff3.exe" -m "$base" "$alocal" "$remote" -o "$result"
else
    #there is not always a common ancestor: DiffMerge needing 3 files, BASE will be the result
    #"C:/Program Files/SourceGear/DiffMerge/DiffMerge.exe" "$alocal" "$result" "$remote" -m --result="$result" --title1="Mine" --title2="Merging to: $result" --title3="Theirs"

    # KDiff3 however does know how to merge based on 2 files (not just 3)
    "C:/Program Files/KDiff3/kdiff3.exe" -m "$base" "$remote" -o "$result"
fi

• Declare your merge wrapper for Git

Git config commands:

git config --global merge.tool diffmerge
git config --global mergetool.diffmerge.cmd "merge.sh \"$PWD/$LOCAL\" \"$PWD/$BASE\" \"$PWD/$REMOTE\" \"$PWD/$MERGED\"
git config --global mergetool.diffmerge.trustExitCode false
git config --global mergetool.diffmerge.keepBackup false

• Check that autoCRLF is false

git config at system level:

git config ---system core.autoCRLF=false

• Test that, when two lines are identical (but their eol chars), both DiffMerge or KDiff3 will ignore those line during a merge.

DOS script (note: the dos2unix command comes from here, and is used to simulate a Unix eol-style. That command has been copied in the directory mentioned at the beginning of this answer.):

C:\HOMEWARE\git\test>mkdir test_merge C:\HOMEWARE\git\test>cd test_merge C:\HOMEWARE\git\test\test_merge>git init C:\HOMEWARE\git\test\test_merge>echo a1 > a.txt & echo a2 >> a.txt C:\HOMEWARE\git\test\test_merge>git add a.txt C:\HOMEWARE\git\test\test_merge>git commit -m "a.txt, windows eol style" C:\HOMEWARE\git\test\test_merge>git checkout -b windows Switched to a new branch 'windows' C:\HOMEWARE\git\test\test_merge>echo a3 >> a.txt & echo a4 >> a.txt C:\HOMEWARE\git\test\test_merge>git add a.txt C:\HOMEWARE\git\test\test_merge>git commit -m "add two lines, windows eol style" C:\HOMEWARE\git\test\test_merge>git checkout master C:\HOMEWARE\git\test\test_merge>git checkout -b unix Switched to a new branch 'unix' C:\HOMEWARE\git\test\test_merge>echo au3 >> a.txt & echo au4 >> a.txt && echo au5 >> a.txt C:\HOMEWARE\git\test\test_merge>dos2unix a.txt Dos2Unix: Processing file a.txt ... C:\HOMEWARE\git\test\test_merge>git add a.txt C:\HOMEWARE\git\test\test_merge>git commit -m "add 3 lines, all file unix eol style" [unix c433a63] add 3 lines, all file unix eol style C:\HOMEWARE\git\test\test_merge>git merge windows Auto-merging a.txt CONFLICT (content): Merge conflict in a.txt Automatic merge failed; fix conflicts and then commit the result. C:\HOMEWARE\git\test\test_merge>git ls-files -u 100644 39b4c894078a02afb9b1dfeda6f1127c138e38df 1 a.txt 100644 28b3d018872c08b0696764118b76dd3d0b448fca 2 a.txt 100644 3994da66530b4df80189bb198dcfac9b8f2a7b33 3 a.txt C:\HOMEWARE\git\test\test_merge>git mergetool Merging the files: a.txt Normal merge conflict for 'a.txt': {local}: modified {remote}: modified Hit return to start merge resolution tool (diffmerge):

At this point (Hitting "return"), DiffMerge or KDiff3 will open, and you will see for yourself what lines are actually merged, and what lines are ignored.

Warning: the result file will always be in Windows eol mode (CRLF) with DiffMerge...
KDiff3 offers to save in one way or another.

Answer 2

I was looking for the same answer and I found out this

Merging branches with differing checkin/checkout attributes

If you have added attributes to a file that cause the canonical repository format for that file to change, such as adding a clean/smudge filter or text/eol/ident attributes, merging anything where the attribute is not in place would normally cause merge conflicts.

To prevent these unnecessary merge conflicts, git can be told to run a virtual check-out and check-in of all three stages of a file when resolving a three-way merge by setting the merge.renormalize configuration variable. This prevents changes caused by check-in conversion from causing spurious merge conflicts when a converted file is merged with an unconverted file.

As long as a "smudge→clean" results in the same output as a "clean" even on files that are already smudged, this strategy will automatically resolve all filter-related conflicts. Filters that do not act in this way may cause additional merge conflicts that must be resolved manually.

So running this command in any repository will do the trick:

git config merge.renormalize true

Answer 3

After reading https://stackoverflow.com/a/12194759/1441706 and https://stackoverflow.com/a/14195253/1441706

for me, this command did the trick perfectly:

git merge master -s recursive -X renormalize

Answer 4

As in this answer: https://stackoverflow.com/a/5262473/943928

You could try: git merge -s recursive -Xignore-space-at-eol

Answer 5

What I did was leave everything as default (i.e. autocrlf=true), touch all files (find . -exec touch {} \;), let git see them as 'modified' and commit them back, and be done with it. Otherwise you'll always either be plagued by annoying messages or surprising differences, or have to turn off all of git's whitespace features.

You'll lose blame information, but it's better to do it sooner rather than later :)

Answer 6

"git merge -Xrenormalize" works like a charm.

Answer 7

It doesn't look like this can be done directly but this post suggests a work around.

http://osdir.com/ml/git/2009-02/msg02532.html

Answer 8

http://stahlforce.com/dev/index.php?tool=remcrlf

I tried it, but if after the last line in your code you didn't already have CRLF it adds by itself a LF and the file looks changed in git. Other than that it works.

Answer 9

It seems to me now that the best way is to normalized the line endings on both branches (and commit) before merging them.

I googled "convert crlf to lf" and found this as the first results:
http://stahlforce.com/dev/index.php?tool=remcrlf

I downloaded it and used, seems like a nice tool.

>sfk remcr . .py

Be sure though to specify a directory and a file type (e.g. .py) otherwise it might try to mess with the contents of the .git directory!

Answer 10

AFAICT, (I haven't tried it) you could use git diff to compare the branch you want to merge to the common ancestor, then apply the results with git apply. Both commands have --ignore-whitespace options to ignore line ending and white space errors.

Unfortunately, if the patch doesn't apply cleanly, the whole operation is aborted. You can't fix merge conflicts. There is a --reject option to leave unpatchable hunks in .rej files, which helps, but isn't the same as having the merge conflicts shown in one file.

Answer 11

After reading Resolve merge conflicts: Force overwrite all files

I finally resolved my version of this issue. I was trying to pull updates from the upstream repo but my current one was having CRLF related issues and was unable to merge as result. It should be noted I had NO LOCAL CHANGES i needed to worry about. The following steps resolved my issue:

As per github's instructions on syncing forks (https://help.github.com/articles/syncing-a-fork/):

1. git fetch upstream

2. git reset --hard upstream/master
   My limited understanding of git tells me this is doing what i want-- rebasing my fork (with no actual uncommitted changes) to gain all the changes made to the upstream source. According to the source page, this step should normally not be required, but the CRLF issue made it required.

3. git merge upstream/master

4. git push

Answer 12

however i suggest to use tool like sed to achieve correct line endings, and then diff files. I spent couple hours on diffing projects with various line endings.

The best way was to:

1. copy only project files (omit .git directory) to another directory create repository in it, then add files and commit them (should be on master branch in new repository).
2. copy files from second project to same folder, but another branch for example dev (git checkout -b dev), commit files on this branch and run (if first project is in master): git diff master..dev --names-only to see names of changed files only