Should I load the whole database at initialization in Flask web application?

Question

I'm develop a web application using Flask. I have 2 approaches to return pages for user's request.

1. Load requesting data from database then return.
2. Load the whole database into python dictionary variable at initialization and return the related page when requested. (the whole database is not too big)

I'm curious which approach will have better performance?

Answer 1

Of course it will be faster to get data from cache that is stored in memory. But you've got to be sure that the amount of data won't get too large, and that you're updating your cache every time you update the database. Depending on your exact goal you may choose python dict, cache (like memcached) or something else, such as tries.

There's also a "middle" way for this. You can store in memory not the whole records from database, but just the correspondence between the search params in request and the ids of the records in database. That way user makes a request, you quickly check the ids of the records needed, and query your database by id, which is pretty fast.