Delay required between a file created via external program using system() and opening it via open()?

Question

I'm trying to create a TAR archive from my program and then opening the archive for further processing. I have a 2 second delay between calling system() and open(). So far, it works fine, but I'm not sure why the 2 second delay is necessary or if it's the correct solution.

Without the delay, I get error code 2 (ENOENT "No such file or directory") from the open() call. My first thought was the filesystem wasn't updating itself fast enough and open() couldn't find the file. But what if the system is really busy? Do I need a longer delay? Should I loop until open() succeeds instead of the delay? Is the problem something completely different?

UPDATE
The root filesystem is EXT2. /tmp is mounted in RAM using TMPFS. I'm using tar to create an archive, not extract contents of one. Essentially, my program is supposed to create an archive of some log files and send them over the network (that's why I open the archive after creating it).

int return_value = system("/bin/tar -czf /tmp/logs.tar.gz /var/log/mylogs.log* &> /dev/null");
// error checks on return_value as described here: http://linux.die.net/man/2/wait
if(return_value != 0) {
  return return_value;
}
//usleep(2000000);
return_value = open("/tmp/logs.tar.gz", O_RDONLY | O_LARGEFILE, 0);
// success or failure depending on whether there's a delay or not

Answer 1

You think you are redirecting tar's output with "&>", but actually you are running it in the background, because system() happens to invoke a shell that doesn't support &> and so interprets it as "&" followed by ">". The delay causes your program to wait long enough that tar completes.

The fix is to modify your command to use syntax that your shell supports. Throwing the error output from tar is probably a mistake in any case.

Answer 2

You could even avoid running an external tar command by using libtar directly in your program.

---

ADDED

And you should show us your program. I'm pretty sure that if the call to system just extracted some file thru tar, it is available just after a successful system call, e.g. something like:

 int err = system("/bin/tar xf /tmp/foo.tar bar");
 int fd = -1;
 if (err == 0)
      fd = open("bar", O_RDONLY);
 // fd is available

there is no reason to wait a few seconds in this code. You are probably doing more complex things, or you forgot to test the result of system

Answer 3

Here's what I would try:

1. fork/exec tar yourself, and have your parent collect the tar-child. If system is introducing a race condition with the file system, taking control of the child process creating/reaping may help.

2. touch an empty file (fopen for writing and close) and then tar into into the new file.

3. Give tar the --verify option; the file has to exist in order to be verified :)