Maininting consistency between neo4j and mysql datastores

Question

I have a mysql database as my basic datastore for the master data. For complex multilevel queries similar to Friends of friends type i am having a graph datastore neo4j. Problem I am facing is while maintaining transactions, where i have to insert a user record in mysql and user node in neo4j. Now i want both of them to be successful. What i have accomplished is if neo4j insert is successful then I insert a mysql user record. but what if insert mysql user record fails i have an inconsistent state. Similar can happen if i insert mysql first and then neo4j. Is there any way i can accomplish transaction across mysql and neo4j. Would i need to maintain some kind of failed transaction log and execute them later?

Thanks! I know many wont agree with the combined approach for neo4j and mysql but i find this the best solution at the moment if i can pull away with consistent datastores.

Answer 1

Polyglot persistence (multi-database systems) is becoming commonplace, and this is a common challenge. You won't find any built-in mechanisms for transactions across disparate databases. Now: While there's no single "right answer" for your scenario, (and I'll do my best to keep my answer objective), think about your System of Record - which database holds the truth and needs to be correct? My guess is, it's the MySQL database.

So: Now you have your MySQL database, properly taken care of, saving content transactionally. Now there's your Neo4j database, which is being used for ancillary functions (searching for friends, in your case). Why not add your Neo4j graph nodes (and relationships) after the fact, as a separate operation? Will it really impact your system's operation if the insert into Neo4j is delayed? Is there anything detrimental to having an eventually-consistent update to your graph database? I suspect that, even with a follow-on operation, the delay will be minimal. That said: Only you know your app, and whether this synchronization between databases needs to be absolute.