Shared Persistent Storage in Python

Question

Several processes are each writing a file to a directory. The goal is to control the size of the directory such that whenever it reaches a size (S), all processes stop writing to the directory and discard the file they are about to write.

If the size then becomes lower than S because some of those files were removed, the processes will resume writing files.

It seems that I need inter-process locking to achieve this design. However, I thought maybe there's an easier way, since inter process locking is not readily available in python and obviously there's contention between processes.

Python 2.7 Platforms (Win, Mac, Linux)

Answer 1

Using lock files may be an option. For example, each process checks for a file like "/target_dir/lock" before write. If file exists, process will not write anything. So you have to run separate monitor process, which checks directory size, and creates or deletes lock file.