What does the 'with' statement do in python? [duplicate]

Question 1

with statements open a resource and guarantee that the resource will be closed when the with block completes, regardless of how the block completes. Consider a file:

with open('/etc/passwd', 'r') as f:
    print f.readlines()

print "file is now closed!"

The file is guaranteed to be closed at the end of the block -- even if you have a return, even if you raise an exception.

In order for with to make this guarantee, the expression (open() in the example) must be a context manager. The good news is that many python expressions are context managers, but not all.

According to a tutorial I found, MySQLdb.connect() is, in fact, a context manager.

This code:

conn = MySQLdb.connect(...)
with conn:
    cur = conn.cursor()
    cur.do_this()
    cur.do_that()

will commit or rollback the sequence of commands as a single transaction. This means that you don't have to worry so much about exceptions or other unusual code paths -- the transaction will be dealt with no matter how you leave the code block.

Question 2

Fundamentally it's a object that demarcates a block of code with custom logic that is called on entrance and exit and can take arguments in it's construction. You can define a custom context manager with a class:

class ContextManager(object):

    def __init__(self, args):
        pass

    def __enter__(self):
        # Entrance logic here, called before entry of with block
        pass

    def __exit__(self, exception_type, exception_val, trace):
        # Exit logic here, called at exit of with block
        return True

The entrance then gets passed an instance of the contextmanager class and can reference anything created in the __init__ method (files, sockets, etc). The exit method also receives any exceptions raised in the internal block and the stack trace object or Nones if the logic completed without raising.

We could then use it like so:

with ContextManager(myarg):
    # ... code here ...

This is useful for many things like managing resource lifetimes, freeing file descriptors, managing exceptions and even more complicated uses like building embedded DSLs.

An alternative (but equivalent) method of construction is to the contextlib decorator which uses a generator to separate the entrance and exit logic.

from contextlib import contextmanager

@contextmanager
def ContextManager(args):
    # Entrance logic here
    yield
    # Exit logic here

Question 3

Think of with as creating a "supervisor" (context manager) over a code block. The supervisor can even be given a name and referenced within the block. When the code block ends, either normally or via an exception, the supervisor is notified and it can take appropriate action based on what happened.