Web Caching Servers for SQL Server OLTP Env. Recommendations

Question

I inhereted a high volume OLTP DB which I have free reign to improve as much as I find reasonably possible. The improvements already were very helpful but I want to take it to the next level. The data access patterns I found made it a good candidate IMO to cache the data on other servers and I would love to hear anyone's experience or recommendations with this type of setup.

We have a DB that gets about 3GB of data added to a table every day and reporting on it used to be very slow. The data does not change once it's put in, and no data gets inserted that is over a week old. Rows inputted within the last 3 days tend to see thousands of inserts between tens of millions of rows.

I was thinking of having data over 2 weeks old get pushed out to MongoDB. I could then have the 2 week sliding window data that is not pushed out to Mongo be, be cached by some kind of caching software so those get queried and displayed instead of the data being read out of the DB the whole time. I figure this way we still get full A.C.I.D compliance by having the DB engine validate all the data, have high read performance as it is not hitting the DB, then Mongo can take it when it is no longer a 'transaction'.

Anyone have any recommended solutions? I was looking at MemCached, but not quite sure if that's a good or even plausible solution. Thanks!

Answer 1

I don't have specific experience with SQL Server, but what you are describing does seem like a valid use case for MongoDB.

Note that while MongoDB can't directly handle transactions, it is capable of handling certain operations in an atomic fashion (see findAndModify, for instance). Additionally, with journaling enabled, you shouldn't have any reason to worry about durability. MongoDB is a reliable data store and will not lose or corrupt your data.

MongoDB itself can also act as a performant cache if you run a second deployment with journaling disabled. In this instance, writes will take place in memory and only be persisted to the disk every 60 seconds (unless otherwise configured). This will provide comparable performance to memcache which is solely in-memory while allowing you to keep your stack a bit simpler.

Hope this helps!

Answer 2

Another thing you could consider is using the new In-Memory OLTP feature in SQL Server 2014. That feature improves efficiency and scaling for OLTP workloads. You will potentially be able to get a lot more out of your existing server, without the need to consider specific caching mechanisms.