Use case for low-level os.open, os.fdopen, and friends?

Question

In Python 3.2 (and other versions), the documentation for os.open states:

This function is intended for low-level I/O. For normal usage, use the built-in function open(), which returns a file object with read() and write() methods (and many more). To wrap a file descriptor in a file object, use fdopen().

And for fdopen():

Return an open file object connected to the file descriptor fd. This is an alias of open() and accepts the same arguments. The only difference is that the first argument of fdopen() must always be an integer.

This comment in a question on the difference between io.open and os.open (this difference is entirely clear to me, I always use io.open, never os.open) asks: why would anyone choose Python for low-level I/O?, but doesn't really get an answer.

My question is very similar to the comment-question: In Python, what is the use case of low-level I/O through os.open, os.fdopen, os.close, os.read, etc.? I used to think it was needed to deamonise a process, but I'm not so sure anymore. Is there any task that can only be performed using low-level I/O, and not with the higher-level wrappers?

Answer 1

Major Differences:

• Low level access to files is unbuffered
• Low level access is not portable
• Low level allows more fine grained control, e.g. whether to block or not to block upon read

Use cases for low level io:

• The file is a block device
• The file is a socket
• The file is a tty
• ...

In all these cases you might wish to have that more fine grained control (over buffering and blocking behavior).

You probably never will need the low level functions for regular files. I think most of the time the use case will be some device driver stuff. However, this would better be done in C. But I can see the use case for python as well, e.g. for fast prototyping of device drivers.

Answer 2

I use it when I need to use O_CREAT | O_EXCL to atomically create a file, failing if the file exists. You can't check for file existence then create the file if your test found that it does not exist, because that will create a race condition where the file could be created in the interim period between your check and creation.

Briefly looking at the link you provided, I do believe pidfile creation has a race condition.

In Python 3.3, there is a new 'x' mode added to open() that seems to do this. I haven't tried it, though.