Catching exceptions by generators inside `for` loop

Question

I want to write decorator for generators that will catch all exceptions inside for loop, process them and continues loop.

I wrote this decorator (for Django 1.5 ORM):

def savepoint_loop(generator, uniq_error_in='_uniq'):
    with commit_on_success():
        sp = savepoint()
        for obj in generator:
            try:
                yield obj
            except DatabaseError as e:
                if uniq_error_in not in e.args[0]:
                    raise
                savepoint_rollback(sp)
                yield None
            else:
                savepoint_commit(sp)
                sp = savepoint()

And I use it like:

loop = savepoint_loop(offer.booking_data.iteritems())
for provider_name, booking_data in loop:
    try:
        BookingData.objects.create(
            offer=pnr_offer, provider=provider_name, **booking_data)
    except Exception as e:
        loop.throw(e)

But it doesn't looks Pythonic. It allows me to make my code DRY, but looks confusing. Is there any way to make it cleaner? At least I want to remove try-except-throw construction or change it to with operator.

Ideally it should look like this:

for provider_name, booking_data in savepoint_loop(
        offer.booking_data.iteritems()):
    BookingData.objects.create(
         offer=pnr_offer, provider=provider_name, **booking_data)

Answer 1

import contextlib

@contextlib.contextmanager
def error_processor(uniq_error_in='_uniq'):
    sp = savepoint()
    try:
        yield
    except DatabaseError as e:
        if uniq_error_in not in e.args[0]:
            raise
        savepoint_rollback(sp)
    else:
        savepoint_commit(sp)

This is a context manager that should do the job your coroutine does, but hopefully in a more understandable manner. You'd use it as follows:

with commit_on_success():
    for provider_name, booking_data in offer.booking_data.iteritems():
        with error_processor():
            BookingData.objects.create(
                offer=pnr_offer, provider=provider_name, **booking_data)

I couldn't fit the commit_on_success into the context manager, as the commit_on_success needs to go around the for loop, but the error handling needs to go inside the loop.

Answer 2

Hm... I think I see why this isn't straightforward. Any loop over an iterable basically comprises the following steps:

1. initialization
2. check termination condition
3. get next element
4. execute body of loop
5. ... repeat steps 2-4 until the termination condition is satisfied

In a Python for loop, steps 2 and 3 are encapsulated in the for x in y statement. But it sounds like you want to use the same try...except to catch exceptions in steps 3 and 4. So it would appear that this will require you to "decompose" the for statement, i.e. implement it manually using a while loop instead.

iterable = offer.booking_data.iteritems() # step 2
try:
    while True:
        try:
            provider_name, booking_data = iterable.next() # step 3
            BookingData.objects.create(...) # step 4
        except:
except StopIteration:
    pass

I'm sure you would agree that this is stylistically worse than the code sample you already have in your question, with try...except inside the for loop. Sure, it's conceivable that it might be useful, if you really need steps 3 and 4 to be inside the same try block, but I would imagine cases like that are rare.

It's also possible there is some sneaky way to construct a generator that does what you want, but I can't think of one.