Python why doesn't writing a contextmanager for an sqlite3 cursor work?

Question

This is supposed to work yet just says no stocks table - supposed lost the connection somewhere inside the contextmanager?

import sqlite3
from contextlib import contextmanager

@contextmanager
def doquery(conn, q, params=()):
    c = conn.cursor()
    c.execute(q, params)
    conn.commit()
    yield c    
    c.close()

with sqlite3.connect(':memory:') as db:    
    doquery(db,'''create table stocks
    (date text, trans text, symbol text,
    qty real, price real)''')

    doquery(db,"""insert into stocks
          values ('2006-01-05','BUY','RHAT',100,35.14)""")

    with doquery(db, 'select * from stocks') as r:
        for row in r:
            print row

Answer 1

The problem is with the way you are using the context manager. Calling doquery simply creates a context manager object - you need to use it within a with statement, which calls its __enter__ and __exit__ methods as appropriate. For example, try the following:

from contextlib import contextmanager

@contextmanager
def enter_exit(text):
    print('entering')
    yield text
    print('exiting')

print(enter_exit('attempt 1'))

with enter_exit('attempt 2') as t:
    print(t)

The output I get is:

<contextlib._GeneratorContextManager object at 0xcf3e90>
entering
attempt 2
exiting

You might want to re-read the documentation about the with statement and contextlib.

Another problem with your code is that if c.execute or conn.commit raises an exception, c.close will not be called - I don't know if that is actually necessary, but presumably it is the reason why you want to use a context manager rather than a function in the first place. The following changes should fix both problems:

import sqlite3
from contextlib import contextmanager

@contextmanager
def doquery(conn, q, params=()):
    c = conn.cursor()
    try:
        c.execute(q, params)
        conn.commit()
        yield c
    finally:
        c.close()

with sqlite3.connect(':memory:') as db:
    with doquery(db,'''create table stocks
                 (date text, trans text, symbol text,
                 qty real, price real)'''):
        pass

    with doquery(db,"""insert into stocks
                 values ('2006-01-05','BUY','RHAT',100,35.14)"""):
        pass

    with doquery(db, 'select * from stocks') as r:
        for row in r:
            print(row)

However, I don't think this is the cleanest way of doing this. As far as I can see, there is no reason to create three separate cursor objects - you can use the same one for each query. I don't think the call to conn.commit is actually necessary either - using the database connection as a context manager will automatically commit transactions, or roll them back if an exception is raised (see the sqlite3 module documentation).

EDIT: Here is a much cleaner version, which still works. I really don't know what closing the cursor actually does though - it probably isn't necessary (Cursor.close doesn't even seem to be documented).

import sqlite3
from contextlib import closing

with sqlite3.connect(':memory:') as db:
    with closing(db.cursor()) as c:
        c.execute('''create table stocks
                 (date text, trans text, symbol text,
                 qty real, price real)''')
        c.execute("""insert into stocks
                 values ('2006-01-05','BUY','RHAT',100,35.14)""")
        c.execute('select * from stocks')
        for row in c:
            print(row)

Answer 2

It seems that yield is interfering with the create table and the insert into statements.

In the following I don't use yield for except for the select and it works ok:

#!/usr/bin/python3
import sqlite3
from contextlib import contextmanager

@contextmanager
def doquery(conn, q, params=()):
    c = conn.cursor()
    c.execute(q, params)
    conn.commit()
    yield c
    c.close()

@contextmanager
def doquery2(conn, q, params=()):
    c = conn.cursor()
    c.execute(q, params)
    conn.commit()
    c.close()

with sqlite3.connect(':memory:') as db:
    doquery2(db,'''create table stocks
    (date text, trans text, symbol text,
    qty real, price real)''')

    doquery2(db,"""insert into stocks                                                                                                                
          values ('2006-01-05','BUY','RHAT',100,35.14)""")

    with doquery(db, 'select * from stocks') as r:
        for row in r:
            print(row[0])