Store data from MongoDB query into object and update every 20 minutes

Question

Please excuse me if this is a newb question as I am a front-end developer attempting to learn full stack.

My node application inserts records into a MongoDB every 20 minutes.

It seems illogical to query the database every time a user visits the page, because the database is identical for a time period of 20 minutes(until I insert new records every 20 minutes).

I'm thinking I could store the results of the query in an object, or something, and update this object every 20 minutes(every time I insert new records, which is a function that runs every 20 minutes).

How would I go about doing this? Would I need this to be a global variable?

Is this a good or bad idea and why?

Answer 1

If your queries are indexed and/or fast, hitting the database is probably the way to go. The other option would be to have a caching layer. If you did that in raw node.js you'd buy yourself a lot of complexity (and possible performance issues), so I'd stay away from that. If you did it in another service (redis, memcached) you probably wouldn't see much performance gain unless the queries are complex or slow. If the queries are complex, you could regenerate the results every 20 minutes and save that somewhere in mongo and just run the queries from that. Ex:

setInterval(regen_stats, 20 * 60 * 1000)

The bottom line is that databases are meant to be queried, so you don't have to start taking things away from them until it stops performing (usually due to scale or complexity of queries)