Putting together a Python context manager: a puzzle

Question

I'm puzzled by how to arrange all the things that Python context managers can do into the appropriate spots.

As I understand it, the elements that can potentially go into building a context manager include:

• A: Something that always happens
• B: Some preparation needed for C
• C: Create and establish an object X used in the context
• D: Do some things that happen using a successfully established X before context start
• E: Return X to the context (for use by as)
• F: Wrap up with X when all is well at the end of the context
• G: Deal the consequences of failure in C and B before entering the context
• H: Deal the consequences of failure in context

I think I roughly get where each of these element goes in a context manager function, but am completely at a loss for how to arrange them in classes.

Is there a template for context manager functions and classes that shows were each of these elements goes in both functions and (especially) classes? I've looked through many examples here and elsewhere, but have found none that is comprehensive, and many that use actual code that I can't always map to each of the building blocks above.

---

I think I basically understand how a context manager behaves when implemented through a function:

from contextlib import contextmanager     
@contextmanager
def log_file_open(oec_data, build_description, log_dir):
    # A: Something that always happens
    try:
        # B: Some stuff needed to make a_thing
        a_thing = establish_thing_in_a_way_that_might_fail() # C
        # D: Some things that happen using a_thing at context start
        yield a_thing # E
        # F: Wrap up with a_thing when all is well
    except:
        # G: Deal the consequences of failure in try or...
        # H: Deal the consequences of failure in context
    finally:
        # Could F go here instead?

For example, to open a file that to which something should be written on successful open and close, but which should be cleaned up if there's a problem, I could write

from contextlib import contextmanager     
@contextmanager
def log_file_open(oec_data, build_description, log_dir):
    print('Entering context...')
    try:
        usable_file_name = get_some_name()
        a_thing =  open(usable_file_name, mode='w')
        a_thing.write('Logging context started.')
        yield a_thing
        a_thing.write('Logging context ended.')
    except:
        a_thing.close()
        os.remove(a_thing.name)
        raise

But I'm not sure this is right, and I'm confused how it maps to the use of __enter()__ and __exit()__ in classes. Is it (schematically):

def __init__(self):
    # A: Something that always happens

def __enter__(self):
    try:
        # B: Some stuff needed to make a_thing
        a_thing = establish_thing_in_a_way_that_might_fail() # C
        # D: Some things that happen using a_thing at context start
     except:
        # G: Deal the consequences of failure in try
        a_thing = some_appropriate_blank_value
     finally:
        return a_thing # E

 def __exit__(self, type, value, traceback):
        if type is None:
            # F: Wrap up with a_thing when all is well
            return True
        else:
            # H: Deal the consequences of failure in context
            return False

Answer 1

You're mixing up error handling in generating the context value and error handling in the context itself. It's far better to write:

@contextmanager
def fn(...):
    value = ...      # A, B, C, D: setup
    try:
        yield value  # E: pass value to client
    except:          # or better, finally:
        ...          # F, H: cleanup

This way you know that you're only dealing with exceptions that originated in client code, and you streamline your cleanup code as you know that the setup succeeded. There's usually no point trying to handle exceptions in the setup code; you don't want client code to have to handle a None context value. This means that __enter__ is simply:

def __enter__(self):
    self.value = ...   # A, B, C, D: setup
    return self.value  # E: pass value to client

If __enter__ raises an exception, then __exit__ will not be called.

Note also that finally is better than except, unless you're planning to suppress exceptions from client code, which is very rarely useful. So __exit__ is simply:

def __exit__(self, type, value, traceback):
    ...                # F, H: cleanup
    return False       # don't suppress any exception

Answer 2

I think your understanding is mostly correct. A context manager is an object, which manages the context through its __enter__ and __exit__ methods. So what happens in __init__ stays true for the life of the object. Let's look at a concrete example:

class CMan(object):
    def __init__(self, *parameters):
        "Creates a new context manager"
        print "Creating object..."

    def __enter__(self):
        "Enters the manager (opening the file)"
        print "Entering context..."
        a_thing = self # Or any other relevant value to be used in this context
        print "Returning %s" % a_thing
        return a_thing

    def __exit__(self, type, value, traceback):
        "Exits the context"
        if type is None:
            print "Exiting with no exception -> Wrapping up"
            return
        print "Exiting with exception %s" % type

Which would be used as this:

>>> with CMan(1,2,3) as x:
...     print 1 + 1
Creating object...
Entering context...
Returning <__main__.CMan object at 0x02514F70>
2
Exiting with no exception -> Wrapping up

Note that creating the object on the fly is not mandatory:

>>> mgr = CMan(1,2,3)
Creating object...
>>> with mgr as x:
...     print 1 + 1
Entering context...
Returning <__main__.CMan object at 0x02514F70>
2
Exiting with no exception -> Wrapping up

Finally, the return value of __exit__ determines whether an exception should be raised. If the value evaluates to False (e.g. False, 0, None...), any exception will be raised. Otherwise, this means the context manager has handled the exception, and it does not need to be raised. For instance:

>>> class Arithmetic(object):
...     def __enter__(self):
...         return self
...     def __exit__(self, type, value, traceback):
...         if type == ZeroDivisionError:
...             print "I dont care -> Ignoring"
...             return True
...         else:
...             print "Unknown error: Panicking !"
...             return False

>>> with Arithmetic() as a:
...     print 1 / 0 # Divide by 0
I dont care -> Ignoring

>>> with Arithmetic() as a:
...     print 1 + "2" # Type error
Unknown error: Panicking !
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
TypeError: unsupported operand type(s) for +: 'int' and 'str'

Note that in the case of the divide by 0 error, as __exit__ returned True, the error is not propagated. In other cases, it is raised after exiting the context manager. You can think of a call to a context manager:

>>> with X as x:
...     f(x)

as being equivalent to:

>>> x = X.__enter__()
>>> try:
...     exc = None
...     f(x)     
... except Exception as e:
...     exc = e
... finally:
...     handled = X.__exit__(exc)
...     if exc and not handled:
...         raise exc

Of course, if an exception is raised inside your method __enter__ or __exit__, it should be handled appropriately, e.g. if generating a_thing could fail. You can find a lot of resources on the web by looking for 'Python with statement', which is usually how you refer to this pattern (although Context manager is indeed more correct)