managing user privileges in sqlalchemy

Question

I have an sqlalchemy script that creates and uses many engine instances representing many user connections. All the engines are configured to point to the same postgres database.

I have one engine, oSuperEngine that can do super stuff. I have another engine 'oBobsEngine` for Bob.

Now I'm doing something like this:

sSQL = "GRANT ALL PRIVILEGES ON TABLE \"NICE_TABLE\" to bob;"
oSuperEngine.execute(sSQL)

sSQL = "insert into \"NICE_TABLE\" (foo) values (bar)"
oBobsEngine.execute(sSQL) # ERROR HERE

And getting:

ProgrammingError: (ProgrammingError) permission denied for relation NICE_TABLE

Why is this?

in psql \dp tells me that bob was never granted permissions. If I log into psql with a super user and grant bob his permissions manually then everything works fine. I use exactly the same commands in this case as I was executing via sqlalchemy.

Is something not flushing correctly? Does sqlalchemy for some reason like to fail silently when attempting grant statements? How can I make this work?

Apologies for not including more code, the code base is moderately convoluted. The order of events has been confirmed to me using logs... please let me know if you need more code in order to understand my problem.

Some logs

2014-03-10 10:07:24,767 - common.sqlalchemy_tools - DEBUG - connection string = "postgresql+psycopg2://super:password@localhost/db_name"
2014-03-10 10:07:24,767 - sqlalchemy.engine.base.Engine - INFO - BEGIN;
2014-03-10 10:07:24,767 - sqlalchemy.engine.base.Engine - INFO - {}
2014-03-10 10:07:24,767 - sqlalchemy.engine.base.Engine - INFO - GRANT ALL PRIVILEGES ON TABLE "MY_TABLE" to bob;
2014-03-10 10:07:24,767 - sqlalchemy.engine.base.Engine - INFO - {}
2014-03-10 10:07:24,768 - sqlalchemy.engine.base.Engine - INFO - COMMIT;
2014-03-10 10:07:24,768 - sqlalchemy.engine.base.Engine - INFO - {}
 ...
2014-03-10 10:07:24,804 - common.sqlalchemy_tools - DEBUG - connection string = "postgresql+psycopg2://bob:password@localhost/db_name"
2014-03-10 10:07:24,814 - sqlalchemy.engine.base.Engine - INFO - BEGIN;
2014-03-10 10:07:24,815 - sqlalchemy.engine.base.Engine - INFO - {}
2014-03-10 10:07:24,827 - sqlalchemy.engine.base.Engine - INFO - insert into "MY_TABLE" (stuff) values (other stuff);
2014-03-10 10:07:24,827 - sqlalchemy.engine.base.Engine - INFO - {}
2014-03-10 10:07:24,828 - sqlalchemy.engine.base.Engine - INFO - ROLLBACK

The first bunch of stuff happens with oSuperEngine, the second bunch with bob's engine. And the error looks like:

ProgrammingError: (ProgrammingError) permission denied for relation MY_TABLE
 'insert into "MY_TABLE" (stuff) values (stuff);' {}

Answer 1

If you execute the script above you run into a problem with transaction isolation. I actually had a similar issue some time ago. The thing to keep in mind here is that the transaction going for Bob does not see changes made by the Super engine because the transaction is still going and the isolation does not allow for an uncommited read. Postgres has extensive documentation about this but the essence is: An uncommited read is not possible in postgres.

And your solution, obviously, is to commit your changes from the Super engine first, then work with it in the other transaction. If have not tried it but my guess is that you have to stay on the default Read committed isolation level (as higher isolation wouldn't allow to detect changes since a transaction - in this case Bobs - has started).

Thus, before executing Bobs query:

oSuperEngine.execute("COMMIT")

However, this raises a side issue that effectively breaks the awesome thing transactions bring you: You cannot roll back easily since the changes are already commited. What you'd basically want here is the opposite of a savepoint:

Whereas a savepoint does not store changes to the database yet, it ensures a rollback would only go back to a particular point, not all the way. You'd want the opposite: Store it in the database but be able to roll back and remove it from it again. I am not aware of such a thing and I highly doubt it exists since it would violate the principles of transaction isolation.

My solution to this problem was to write custom rollback routines that I would call on an exception so that I could manually undo the changes. This is quite a bit of work though and is high-maintenance in the long run. In the end, in my case, I reconsidered my solution and dropped the multiple-engine approach (but that is a decision you have to make yourself).

Answer 2

I had the exact same problem. Fixed by doing like this:

sSQL = "GRANT ALL PRIVILEGES ON TABLE \"NICE_TABLE\" to bob;"
with oSuperEngine.begin() as conn:
    conn.execute(sSQL)

http://docs.sqlalchemy.org/en/latest/orm/session_basics.html#adding-additional-configuration-to-an-existing-sessionmaker