Nesting 'WITH' statements in Python

Question

It turns out that "with" is a funny word to search for on the internet.

Does anyone knows what the deal is with nesting with statements in python?
I've been tracking down a very slippery bug in a script I've been writing and I suspect that it's because I'm doing this:

with open(file1) as fsock1:
    with open(file2, 'a') as fsock2:
        fstring1 = fsock1.read()
        fstring2 = fsock2.read()

Python throws up when I try to read() from fsock2. Upon inspection in the debugger, this is because it thinks the file is empty. This wouldn't be worrisome except for the fact that running the exact same code in the debugging interperter not in a with statement shows me that the file is, in fact, quite full of text...

I'm going to proceed on the assumption that for now nesting with statements is a no-no, but if anyone who knows more has a different opinion, I'd love to hear it.

Answer 1

I found the solution in python's doc. You may want to have a look at this (Python 3) or this (Python 2)

If you are running python 2.7+ you can use it like this:

with open(file1) as fsock1, open(file2, 'a') as fsock2:
    fstring1 = fsock1.read()
    fstring2 = fsock2.read()

This way you avoid unnecessary indentation.

Answer 2

AFAIK you can't read a file open with append mode 'a'.

Answer 3

Upon inspection in the debugger, this is because it thinks the file is empty.

I think that happens because it can't actually read anything. Even if it could, when you append to a file, the seek pointer is moved to the end of the file in preparation for writing to occur.

These with statements work just fine for me:

with open(file1) as f:
    with open(file2, 'r') as g:   # Read, not append.
        fstring1 = f.read()
        fstring2 = g.read()

Note that use of contextlib.nested, as another poster suggested, is potentially fraught with peril here. Let's say you do this:

with contextlib.nested(open(file1, "wt"), open(file2)) as (f_out, f_in):
   ...

The context managers here get created one at a time. That means that if the opening of file2 fails (say, because it doesn't exist), then you won't be able to properly finalize file1 and you'll have to leave it up to the garbage collector. That's potentially a Very Bad Thing.

Answer 4

There is no problem with nesting with statements -- rather, you're opening file2 for append, so you can't read from it.

If you do dislike nesting with statements, for whatever reason, you can often avoid that with the contextlib.nested function. However, it won't make broken code (e.g., code that opens a file for append and then tries to read it instead) work, nor will lexically nesting with statements break code that's otherwise good.

Answer 5

As for searching for "with", prefixing a word with '+' will prevent google from ignoring it.