How do I keep NUL bytes from appearing throughout my GIT repository and commit messages?

Question

I have a relatively clean install of Mac OS X Mavericks 10.9 running git version 1.8.5.1; my files are on a mounted SMB share, and I am no longer able to work properly with my GIT repositories because NULL bytes (\0's) keep appearing in all of my version-controlled files... and even in files generated on the fly by GIT.

Typing git commit -m "Test message" by hand, gives me:

error: a NULL byte in commit log message not allowed.
fatal: failed to write commit object

And if I attempt to git commit instead, the default commit message that appears in the pico view looks like this:

# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# On branch feature/centos_support
# Your branch is up-to-date with 'origin/feature/centos_support'.
#
# Changes to be committed:
#       modified:   README.md
#
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@

For a while, manually removing those NULL bytes at the end worked, and I was able to commit, but at intervals even this no longer works (the error is the same as above), and I am forced to trash my entire repo, re-clone it, and redo whatever I was trying to commit.

The fresh repository works as normal for a while, but eventually the NULL characters creep back in, and I have the same problem again.

The same NULL bytes (as above) are appearing at the start and end of files I modify, causing GIT to treat them as binary files until I manually use vi to delete the NULL rows.

• Has anybody else experienced this, and/or found a solution?
• What could be causing it?

I would appreciate any help you can offer.

Answer 1

It looks like this is a problem with OS X 10.9 Mavericks and (some?) SMB shares. There is currently no fix or workaround.

Some people have suggested connecting to the share with cifs:// instead of smb:// however this does NOT appear to solve the issue.

The issue can be reproduced as follows:

<on an SMB share>

$ echo "blah blah blah blah" > test.txt
$ more test.txt

<file is fine>

$ echo "blah" > test.txt
$ more test.txt

<file has ^@ characters at the end>

So, using the terminal, if you overwrite a file with a smaller one, then you get these NUL characters at the end. There may be other ways to cause the file to be appended or prepended with those NUL characters too.

A number of people are complaining about this, for example: https://netbeans.org/bugzilla/show_bug.cgi?id=237766

Here is the conclusion from that page:

I can replicate the null characters in Komodo Edit and Aptana Studio, so this doesn't appear to be a Netbeans specific issue, and is probably related to the NAS/SMB issue.

I forced the smb1 connection but it made no difference. Even if I eject the shares, and reconnect with cifs:// and create, modify and save a file, it's still corrupted.

It looks like Apple is going to have to resolve this, unless someone can come up with a viable workaround.

Hopefully Apple will fix this issue shortly. Until then, you could potentially use Double Commander or some other program that implements its own SMB stack.

Answer 2

I'm having the same problem since updating to Mavericks. It happens with git repositories stored on SMB drives at every commit that has a shorter message than the previous commit:

$ git commit -a -m "long comment"

[master 2516835] long comment
 1 file changed, 1 insertion(+), 1 deletion(-)

$ git commit -a -m "short"

error: a NUL byte in commit log message not allowed.
fatal: failed to write commit object

I noticed the commit message gets stored in the file COMMIT_EDITMSG, that is were the null bytes (as described by the other posters) end up. Just deleting the file before the commit did the trick:

$ rm .git/COMMIT_EDITMSG 
$ git commit -a -m "short"

[master e8bd92e] short
 1 file changed, 1 insertion(+), 1 deletion(-)

Hope that solution works for you guys too. It was driving me crazy.

Answer 3

Git assumes an UTF-8 encoding for commit messages, I guess this is somehow different on your system, and that's why it sees the \0 byte. Try the following:

git config --add i18n.commitencoding ISO-8859-1
git commit -m "Test message"

Answer 4

Not sure about it, Is it possible that a commit-msg hook is causing this behaviour?

Although you say that you re-clone the repository the hook can be regenerated from inside the repo.

Answer 5

I've experienced this exact problem using Mercurial over a SMB share rather than Git. It was a clean install of Mavericks but connecting to an oldish VM running Ubuntu - which I had connected to for the last couple of years from Snow Leopard without any issues. The VM contains a local development server.

Whenever I tried to commit a file that had got smaller it would pad out the end of the file with empty bytes which caused Mercurial to see the file as a binary file (I believe Mercurial was reading them as UTF-16 but I'm not 100% certain)

It only affects some SMB shares and eventually I managed to track the problem down to the smb.conf file.

I solved the issue by replacing the conf file with a working file from another VM and updating the paths and users.

When I run testparm /etc/samba/smb.conf this is the output of my working file:

[global]
server string = Samba Server Version %v
log file = /var/log/samba/log.%m
max log size = 50
idmap config * : backend = tdb
allocation roundup size = 0
cups options = raw

[webroot-c6]
comment = Webroot on Cent OS 6
path = /var/www/vhosts
force user = admin
force group = apache
read only = No
force create mode = 0755
force directory mode = 0755
case sensitive = Yes
follow symlinks = No
delete readonly = Yes

I still don't know exactly which settings were the problem - if I have the time to find out I'll update this post.

Along the way I tried out all everything like using getting Mavericks to use smb1 rather than the smb2 default, but switching to smb1 made no difference - it was updating smb.conf that fixed the problem and I am now using smb2.

Answer 6

I had been using OSX El Capitan with an SMB share from CentOS fine for a long time and all of a sudden I started getting NUL characters at the end of my files when I shortened them and saved them. I looked through different fixes. What ended up fixing the issue was to re-install SMB using the code below.

yum install samba samba-client samba-common

After that I rebooted and everything started working properly again. I don't know where the change came from OSX or CentOS. This resolved it for me.

Answer 7

Disable ‘unix extensions = off’ option in your Samba configuration file:

#unix extensions = off

(it works on my debian-machine with Samba 2:3.5.6)