Why reading and writing the same file through I/O redirection results in an empty file in Unix?

Question

If I redirect output of a command to same file it reads from, its contents is erased.

sed 's/abd/def/g' a.txt > a.txt

Can anyone explain why?

Answer 1

The first thing the redirection does is to open the file for writing, thus clearing any existing contents. sed then tries to read this empty file you have just created, and does nothing. The file is then closed, containing nothing.

Answer 2

The redirection operations <, >, etc. are handled by the shell. When you give a command to the shell that includes redirection, the shell will first open the file. In the case of > the file will be opened for writing, which means it gets truncated to zero size. After the redirection files have been opened, the shell starts a new process, binding its standard input, output, and error to any possible redirected files, and only then executes the command you gave. So when the sed command in your example begins execution, a.txt has already been truncated by the shell.

Incidentally, and somewhat tangentially, this is also the reason why you cannot use redirection directly with sudo because it is the shell that needs the permissions to open the redirection file, not the command being executed.

Answer 3

You need to use the -i option to edit the file in place:

sed -i .bck 's/abd/def/g' a.txt

EDIT: as noted by neil, the redirection first opens the file for writing thus clears it.

EDIT2: it might be interesting for some reader

On OSX, if you want to use -i with an empty extension to prevent the creation of a backup file, you need to use the -eswitch as well otherwise it fails to parse the arguments correctly:

sed -i -e 's/abc/def/g' a.txt

Answer 4

stdout and stderr will first prepared and then stdin and then the command execute. so a.txt would be clear for stdout first and then when the comamnd execute no content could be found. try sed -i 's/abd/def/g' a.txt or sed 's/abd/def/g' a.txt 1<> a.txt