How to transport and index Cassandra data on Elastic Search?

Question

I'm starting a nodejs application where I want to index Cassandra data on Elastic Search, but what would be the best way to do that?, I gave a look to Storm to accomplish just that but doesn't seem to be the solution. Primarily, I was thinking to use one client for Cassandra and one client for Elastic Search and apply inserts/updates/deletes twice on my application, being one per client, but doesn't appear to be the way to go, and I'm worried about the consistency of this. There's a better way to transport Cassandra data to be indexed on Elastic Search? Storm would help me to accomplish that? Could someone suggest any techniques to transport one database data to another? I'm in a really doubt here with nowhere to look.

Answer 1

Do you want to move the data from Cassandra to ElasticSearch once and only once? Or you want to keep them in sync?

In both cases, I think Storm is a good fit. I used in the past to move data from our RDBMS into Apache Solr. One thing to keep in mind is the limit of writes that Solr/Elastic search can do. If you increased the parallelism, then you are bringing them to the knees.

Another option could be Apache Hadoop but it is only suitable for one time copying or if you want to copy the data (same data of yesterday + what could be new) every day.