How to rewrite a file from a shell script without any danger of truncating the file if out of disk space?

Question

How to rewrite a file from a shell script without any danger of truncating the file if out of disk space?

This handy perl one liner replaces all occurrences of "foo" with "bar" in a file called test.txt:

perl -pi -e 's/foo/bar/g' test.txt

This is very useful, but ...

If the file system where test.txt resides has run out of disk space, test.txt will be truncated to a zero-byte file.

Is there a simple, race-condition-free way to avoid this truncation occuring?

I would like the test.txt file to remain unchanged and the command to return an error if the file system is out of space.

Ideally the solution should be easily used from a shell script without requiring additional software to be installed (beyond "standard" UNIX tools like sed and perl).

Thanks!

Answer 1

From perldoc perlrun:

-i[extension]

specifies that files processed by the "<>" construct are to be edited in-place. It does this by renaming the input file, opening the output file by the original name, and selecting that output file as the default for print() statements. The extension, if supplied, is used to modify the name of the old file to make a backup copy, following these rules:

If no extension is supplied, no backup is made and the current file is overwritten.

[…]

Rephrased:

1. The backup filename is determined from the value of the -i-switch, if one is given.
2. The original file is renamed to the new filename, and opened for the script. Renaming is atomic on most filesystems.
3. A file with the name of the original file is opened for writing. The file will start with length zero, but is not identical to the original file (which has a different name now).
4. After the script has finished, and if no explicit backup extension was provided, the backup file is deleted. The original file is then lost.

Should the system run out of drive space, then the new file is endangered, not the original file which was never copied or moved (at least on filesystems with an inode-like concept).

Answer 2

In general, this can’t be done. Remember that the out-of-space condition can hit anywhere along the sequence of actions that give the appearance of in-place editing. Once the filesystem is full, perl may not be able to undo previous actions in order to restore the original state.

A safer way to use the -i switch is to use a nonempty backup suffix, e.g.,

perl -pi.bak -e 's/foo/bar/g' test.txt

This way, if something goes wrong along the way, you still have your original data.

If you want to roll your own, be sure to check the value returned from the close system call. As the Linux manual page notes,

Not checking the return value of close() is a common but nevertheless serious programming error. It is quite possible that errors on a previous write(2) operation are first reported at the final close(). Not checking the return value when closing the file may lead to silent loss of data. This can especially be observed with NFS and with disk quota.

As with everything else in life, leave yourself more margin for error. Disk is cheap. Dig out the pocket change from your couch cushions and go buy yourself half a terabyte or so.