right persistence technology [closed]

Question

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.

Closed 9 years ago.

We have an application server running in nodejs and we are trying to replace our current persistence technology (MySQL). We are considering several nosql options like CouchDB, MongoDB, Redis, Riak etcetera. The size of the data will not be huge (1 Gb max) and a single server solution is fine.

We are developing our own transformation/query language and for this to work efficiently, the most important requirement of our persistence technlogy is it is very fast in navigating over the datastructure; retrieve a record/document/node, look up the records it links to and then retrieve some of them. This is done recursively in 1 transformation, and a typical transfomation will recursively retrieve about a 100.00 records.

I see 3 possible solution directions

1) Use a NoSQL product as a simple key value store, don't use any of their advanced features, like aggregation, filtering etcetera (except for indexing perhaps) and send a request for every record 1 by 1.

2) Use an in process key value store (Alfred?) (I would think that this is a lot more efficient, but that's just gut feeling, no actual measuring done yet).

3) Our transformation engine is written in javascript and a transformation itself is defined as a complex JSON structure, so we might store the transformation engine as a view in a nosql product that supports Javascript (like MongoDB) and every time we want to do a transformation, we send the JSON structure and let the nosql server do the transformation.

What is the best direction/product you can think of to build this?

gr,

Coen

Answer 1

Couchbase wasn't on your list but would satisfy your requirements, Key/Value JSON data storage, extremely fast and scalable performance. Acts like memcached (RAM speed performance) but with persistence backend. It in fact evolved from memcached -> membase server (distributed memcached with persistence) -> Couchbase (lots of new features including indexing and search). Can scale to millions of ops/second. Open Source with many large customers (i.e. Orbitz, Zynga, LinkedIn) and enterprise support.

http://www.couchbase.com/