How to ensure that a process reads file only when it is available in whole?

Question

I have following requirement.

One process (Process-1) reads from DB, gets the files and places the retrieved files, say in location /process1/data; and eventually places the files in /application/shared_data

Second process (Process-2) will check the data from shared directory /application/shared_data.

These two processes runs continuously, independent of each other.

When Process-1 places a file, say 1.zip from /process1/data to /application/shared_data; chances are that at the same time process-2 tries to read the file that moment itself; so how can we prevent this kind of situation.

That is process-2 should read a given file, ONLY when process-1 has completely placed the files in shared directory.

Any help appreciated.

Regards, Vipin

Answer 1

Either have the two processes send messages to each other, or place the file under a temporary name (in the target directory) first, then move it to its final location. POSIX guarantees that moving a file is an atomic operation.

E.g.

temp=$(mktemp $(dirname $target).XXXXXX)
cp "$source" "$temp"
mv "$temp" "$target"