Using rpyc to connect to database once and serve multiple queries

Question

Trying to serve database query results to adhoc client requests, but do not want to open a connection for each individual query. I'm not sure if i'm doing it right.

Current solution is something like this on the "server" side (heavily cut down for clarity):

import rpyc
from rpyc.utils.server import ThreadedServer
import cx_Oracle

conn = cx_Oracle.conect('whatever connect string')
cursor = conn.cursor()

def get_some_data(barcode):
    # do something
    return cursor.execute("whatever query",{'barcode':barcode})

class data_service(rpyc.Service):
   def exposed_get_some_data(self, brcd):       
       return get_some_data(brcd)


if __name__ == '__main__':
   s = ThreadedServer(data_service, port=12345, auto_register=False)
   s.start()

This runs okay for a while. However from time to time the program crashes and so far i haven't been able to track when it does that.

What i wish to confirm, is see how the database connection is created outside of the data_service class. Is this in itself likely to cause problems?

Many thanks any thoughts appreciated.

Answer 1

I don't think the problem is that you're creating the connection outside of the class, that should be fine.

I think the problem is that you are creating just one cursor and using it for a long time, which as far as I understand is not how cursors are meant to be used.

You can use conn.execute without manually creating a cursor, which should be fine for how you're using the database. If I remember correctly, behind the scenes this creates a new cursor for each SQL command. You could also do this yourself in get_some_data(): create a new cursor, use it once, and then close it before returning the data.

In the long run, if you wish your server to be more robust, you'll need to add some error-handling for when database operations fail or the connection is lost.

A final note: Essentially you've written a very basic database proxy server. There are probably various existing solutions for this already, which already handle many issues you are likely to run in to. I recommend at least considering using an existing solution.