MySQL dump structure of all tables and data of some

Question

I'm trying to dump the structure of all the tables in our database, and then only the data of the ones I specifically want, but i seem to be doing something wrong as I'm not getting the empty tables created for the ones I exclude from the data dump.

I have a text file which specifies which tables I want to dump the data for (called showtables.txt):

SHOW TABLES FROM mydb
WHERE Tables_in_mydb NOT LIKE '%_history'
AND Tables_in_mydb NOT LIKE '%_log';

I am then doing this command to dump the structure of all tables, and then the data of the tables returned by that query in the text file:

mysqldump -u root -pmypassword mydb --no-data > mydump.sql; mysql -u root -pmypassword < showtables.txt -N | xargs mysqldump mydb -u root -pmypassword > mydump.sql -v

I am getting the dump of all the tables included in the results of the showtables query, but I am not getting the structures of the rest of the tables.

If I run just the structure part as a single command, that works fine and I get the structures dumped for all tables. But combining it with the data dump seems to not work.

Can you point me to where I'm going wrong with this?

Thanks.

Answer 1

I think you've got the order of your commandline arguments wrong (the redirection to a file should be the end), and you need an extra parameter for xargs so we can specify the database name to mysqldump.

Additionally, you need to append >> the dump data, otherwise you'd be overwriting the mydump.sql file for each table:

mysqldump -u root -pmypassword mydb --no-data > mydump.sql
mysql -u root -pmypassword -N < showtables.txt | xargs -I {} mysqldump -v -u root -pmypassword mydb {} >> mydump.sql

Sources: http://www.cyberciti.biz/faq/linux-unix-bsd-xargs-construct-argument-lists-utility/

Answer 2

Working off of Jon's answer, but the -I in xargs will run a separate mysqldump command for each table. Easier to just allow the xargs default which appends the output of the previous command to the next command. mysqldump's last argument is a list of all tables you'd like to dump.

My solution also shows connecting through a bastion host. gzip'ing before streaming over the SSH connection is vastly faster than sending the uncompressed SQL over the wire.

FILE=~/production.sql.gz
HOST=ext-db-read-0.cdzvblmx0n9h.us-west-1.rds.amazonaws.com
USER=username
PASS="s3cret"
DB=myapp_prod
EXCLUDE="'activities', 'changelogs'"

ssh bastion.mycompany.com <<EOF > $FILE
    mysqldump -h $HOST -u $USER -p$PASS $DB --no-data | gzip
    mysql -h $HOST -u $USER -p$PASS -N -e "SHOW TABLES WHERE Tables_in_$DB NOT IN ($EXCLUDE)" $DB | xargs mysqldump -v -h $HOST -u $USER -p$PASS $DB | gzip
EOF

If you don't want to save .gz just pipe it through gzip -d:

ssh bastion.mycompany.com <<EOF | gzip -d > $FILE etc

or directly to your local db:

ssh bastion.mycompany.com <<EOF | gzip -d | mysql -uroot myapp_development