Synchronising data between different databases

Question

I'm looking for a possible solution for the following problem.

First the situation I'm at:
I've 2 databases, 1 Oracle DB and 1 MySQL DB. Although they have a lot of similarities they are not identical. A lot of tables are available on both the Oracle DB and the MySQL DB but the Oracle tables are often more extensive and contain more columns. The situation with the databases can't be changed, so I've to deal with that.

Now I'm looking for the following:
I want to synchronise data from Oracle to MySQL and vice versa. This has to be done real time or as close to real time as possible. So when changes are made at one DB they have to be synced to the other DB as quickly as possible.

Also not every table has to be in sync, so the solution must offer a way of selecting which tables have to be synced and which not.

Because the databases are not identical replication isn't an option I think. But what is? I hope you guys can help me with finding a way of doing this or a tool which does exactly what I need. Maybe you know some good papers/articles I can use?

Thanks!

Answer 1

I read your question and your answer. I have worked on both Oracle, SQL, ETL and data warehouses and here are my suggestions:

1. It is good to have a readymade ETL tool. But, if your application is big enough to make you need a tailor made ETL tool, I suggest you for a home-made ETL process.

   • If your transactional database is on Oracle, you can have triggers set up on the key tables that would further trigger an external procedure written in C, C++ or Java.
   • The reason behind using an external procedure is to be able to communicate with both databases at a time - Oracle and MySQL.
   • You can read more about Oracle External Procedures here.

2. If not through ExtProc, you can develop a separate application in Java or .Net that would extract data from the first database, transform it according to your business rules and load it into your warehouse.

In either approaches that you choose, you will have greater control on the ETL process if you implement your own tool, rather than going for a readymade tool.

Answer 2

Thanks for the comments.

I did some further research on ETL and EAI. I found out that I am searching for an ETL tool.