replacing fixed amount of text in a large file

Question

I'm trying to replace a small amount of text on a specific line of a large log file (totaling ~40 mil lines):

sed -i '20000000s/.\{5\}$/zzzzz/' log_file

The purpose of this is to "mark" a line with an expected unique string, for later testing.

The above command works fine, but in-place editing of sed (and perl) creates a temp file, which is costly.

Is there a way to replace a fixed number of characters (i.e. 5 chars with 5 other chars) in a file without having to create a temp file, or a very large buffer, which would wind up becoming a temp file itself.

Answer 1

You could use dd to replace some bytes in place:

dd if=/dev/zero of=path/to/file bs=1 count=10 conv=notrunc skip=1000

would write 10 zeros (0x00) after the 1000s byte. You can put whatever you want to replace inside a file and write the path to it in the if parameter. Then you had to insert the size of the replacement file into the count parameter, so the whole file gets read.

the conv=notrunc parameter tells dd to leave the end of the file untruncated.

This should work well for any 1-byte file encoding.

Answer 2

ex is a scriptable file editor, so it will work in-place:

ex log_file << 'END_OF_COMMANDS'
20000000s/.\{5\}$/zzzzz/
w
q
END_OF_COMMANDS