Making new MYSQL replication

Question

I need to make working mysql replication from master to slave. (tried it once already)

The database is quite large (over 100GB) and it will take some hours to make it ready for new slave.

The database has MyIsam and innoDB engine and both are being written I think my only choice is to copy the data files from master to a new slave? (or make a database dump which im referring later in the topic of ROUND 2)

Before that I have to run down all the services which uses the database and make writelock for tables or should i shut down the whole database?

After data directory sync to the new replication server I started it up and the database with the tables was there. First error that I got rid off by changing bin.log to 007324 and position to 0.

Error 1:

140213 4:52:07 [ERROR] Got fatal error 1236: 'Could not find first log file name in binary log index file' from master when reading data from binary log 140213 4:52:07 [Note] Slave I/O thread exiting, read up to log 'bin-log.007323', position 46774422

After that I got new problems from database and this error came out from every table.

Error 2:

Error 'Incorrect information in file: './database/table.frm'' on query. Default database: 'database'.

Seems that something went wrong.

ROUND 2!

After this scene I started to think that can this be done without long service break.

Master database has been already configured and it works ok to another slave.

So i did some googling and this is what i came up with.

Making read lock to tables:

FLUSH TABLES WITH READ LOCK;

Taking dump:

mysqldump --skip-lock-tables --single-transaction --flush-logs  --master-data=2 -A  > dbdump.sql

Packaging and moving: gzip (pigz) the the dbdump and moving it to slave server after that finding the MASTER_LOG_FILE and MASTER_LOG_POS from the dump.

After that i don't think that i want to import the dbdump.sql because its over 100GB and will take time. So i think SOURCE would be ok option for it.

On SLAVE server:

    CREATE DATABASE dbdump;
     USE dbdump;
     SOURCE dbdump.db;
    
     CHANGE MASTER TO MASTER_HOST='x.x.x.x',MASTER_USER='replication',MASTER_PASSWORD='slavepass',
     MASTER_LOG_FILE='mysql-bin.000001',MASTER_LOG_POS=X;
    
    start slave;

SHOW SLAVE STATUS \G

I haven't tested this yet, am I on to something?

--bp

Answer 1

Realize that issuing a SOURCE command is the same as running an import of the dumped SQL from shell. Either way, it is going to take a long time. Outside of that, you have the steps correct - flush table with read lock on master, make a database dump of master, make sure you note master binlog coordinates, import dump on slave, set binlog coordinates, start replication. Do not work with the raw binaries unless you REALLY know what you are doing (especially for INNODB tables).

If you have a number of large tables (i.e. not just one big one), you could consider parallelizing your dumps/imports by table (or groups of tables) to speed things along. There are actually tools out there to help you do this.

You CAN work with the raw binaries, but it is not for the faint of heart. In the past, I have used rsync to differentially update the raw binaries between master and slave (you still must use flush table with read lock and gather master binlog coordinates before doing this). For MyISAM tables this works pretty well actually. For InnoDB, it can be more tricky. I prefer to use the option to set InnoDB to write index and data files per table. You would need to rsync the ibdata* files. You would delete ib_logfile* files from slave.

This whole thing is a bit of a high wire act, so I would not resort to doing this unless you have no other viable options. Absolutely take a traditional SQL dump before even thinking about attempting a binary file sync, and each time until you are VERY comfortable that you actually know what you are doing.