tungsten replicate from mysql slave to vertica - does not seem to look at relay-log?

Question

I am playing with mysql and vertica and have a master/slave mysql pair that is running with mixed binlog format (default settings).

To replicate this to the vertica database for some analysis tasks, I was considering tungsten.

The master is busy with read/write requests from the app, and the slave is just recording whats happening, so ideally I want to run the tungsten replicator on the mysql slave as it is only reading as well.

Tungsten requires the binlog to be in row based format, and so I enabled a row based binlog on the slave, which creates the binlog and the relay-binlog files. So the slave reads mixed binlog from the master, applies to its database and also keeps the changes in the relay-binlog in row based format. Everything fine so far.

When I use the standard -a --master-slave argument it appears to be only looking at the binlogs that only contain things that happened locally on the slave, but not the relay-binlogs.

The documentation and posts I found all point to running tungsten on the master, and to use it for replication throughout without using mysql master/slave, rather than that hybrid mode I have in mind.

However I would prefer to keep whats working and just add tungsten as a bridge replicating from mysql slave to vertica.

How would I configure that?

How do I tell it what tables I want to replicate from the slave to vertica, rather than replicate all?

Configuration detail:

/etc/my.cnf of slave:

datadir=/var/lib/mysql
log-bin=/var/log/mysql/mysql-bin.log
expire_logs_days=7

# for tungsten replicator we need row based binlogs on the slave
binlog-format=row
## Server timezone is GMT.
#default-time-zone='+00:00'
# Tables default to UTF8.
character-set-server=utf8
collation-server=utf8_general_ci

server-id=3

The script I use to stage tungsten:

tungsten-replicator-2.1.2-30/tools/tungsten-installer --master-slave -a \
  --service-name=mysql2vertica \
  --master-host=testslave1 \
  --cluster-hosts=testslave1 \
  --datasource-user=tungsten \
  --datasource-password=supersecret \
  --home-directory=/opt/tungsten \
  --buffer-size=1000 \
  --java-file-encoding=UTF8 \
  --java-user-timezone=CDT \
  --mysql-use-bytes-for-string=false \
  --svc-extractor-filters=colnames,pkey \
  --property=replicator.filter.pkey.addPkeyToInserts=true \
  --property=replicator.filter.pkey.addColumnsToDeletes=true \
  --start-and-report

It seems to start fine, says its online, but only copies what is in binlog but not relay-binlog.

Answer 1

We're successfully running Tungsten on our MySQL slave, so what you're looking to do is definitely possible.

By default the slave does not copy incoming changes to the outgoing binlog where Tungsten is expecting them. Try adding:

log-slave-updates=TRUE

to your my.cnf file.