Why won't Mercurial merge my XML files with Beyond Compare?

Question

We have a bunch of XML files in our Mercurial repository. When we merge changes in those files from one branch to another, Mercurial complains that some of our XML files are binary, and asks me to choose between the local and other versions of the file.

I've discovered that Mercurial thinks any file with a NULL byte in it is binary, and some of our XML files are encoded as UTF-16, so contain a lot of NULL bytes. My default merge tool is Beyond Compare, which can handle UTF-16 encoded files. I've configured Mercurial as recommended on the Beyond Compare website, but during the merge, Mercurial says:

tool bcomp can't handle binary

What gives? Why does Mercurial think Beyond Compare can't merge my file?

Answer 1

The recommended Mercurial configuration to use Beyond Compare as a merge program looks like this:

[merge-tools]
bcomp.executable = C:\Program Files\Beyond Compare 3\BComp.exe
bcomp.args = $local $other $base $output
bcomp.priority = 1
bcomp.premerge = True
bcomp.gui = True

[ui]
merge = bcomp

According to the Mercurial docs on merge tools, there is a binary option that is False by default. When binary=True, it tells Mercurial the tool can merge binary files. When I turn that option on, however, Mercurial still won't let Beyond Compare merge the XML files.

A little more research turns up the premerge option. When premerge=True, Mercurial merges internally first, and use the merge tool only if there are conflicts to be resolved. Unfortunately, Mercurial always fails to merge a binary file. When a pre-merge fails, Mercurial stops the merge for that file and never launches the merge tool.

The solution would seem to be, then, to also set the premerge option to False. Unfortunately, the pre-merge is very helpful because 90% of the time, there aren't any conflicts, and never see your merge tool. With premerge set to False, every time a merge is needed, Beyond Compare opens. This gets very tedious.

To solve this, I created a second Beyond Compare merge tool configuration which gets used to merge binary files. I set its priority lower than the default bcomp tool, so it only gets called when the main bcomp tool fails, which is usually just for binary files. I added these lines to the merge-tools section of my mercurial.ini file:

bcompbin.executable = C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
bcompbin.args = $local $other $base $output
bcompbin.gui = True
bcompbin.premerge = False
bcompbin.binary = True

One of the side-effects of this is that Beyond Compare will launch to merge binary files for which it can't handle. I haven't encountered that situation yet, so don't know how to handle it. I would probably just temporarily comment out my bcompbin merge tool.