MySQL replication slave error 22

Question

I'm having trouble with MySQL replication on the slave side with error 22, specifically about charset error.

The error stated that it cannot found charset '#45' which I find odd since the query used is stating DEFAULT CHARACTER SET latin1 COLLATE latin1_swedish_ci and I did checked on charset\Index.xml and it did exists.

Master : Server version: 5.5.30-log

Slave : Server version: 5.1.66-log

Current replication status:

mysql> show slave status \G;
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 10.10.2.21
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000024
          Read_Master_Log_Pos: 1065715871
               Relay_Log_File: mysqld-relay-bin.000029
                Relay_Log_Pos: 86980698
        Relay_Master_Log_File: mysql-bin.000024
             Slave_IO_Running: Yes
            Slave_SQL_Running: No
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table:
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 22
                   Last_Error: Error 'Character set '#45' is not a compiled character set and is not specified in the '/usr/share/mysql/charsets/Index.xml' file' on query. Default database: 'db_businesslounge-backup'. Query: 'CREATE DATABASE IF NOT EXISTS `db_businesslounge-backup` DEFAULT CHARACTER SET latin1 COLLATE latin1_swedish_ci'
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 86980553
              Relay_Log_Space: 1065720589
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 22
               Last_SQL_Error: Error 'Character set '#45' is not a compiled character set and is not specified in the '/usr/share/mysql/charsets/Index.xml' file' on query. Default database: 'db_businesslounge-backup'. Query: 'CREATE DATABASE IF NOT EXISTS`db_businesslounge-backup` DEFAULT CHARACTER SET latin1 COLLATE latin1_swedish_ci'

MySQL's charset\Index.xml :

<?xml version='1.0' encoding="utf-8"?>

<charsets max-id="99">

<copyright>
  Copyright (c) 2003, 2012,  Oracle and/or its affiliates. All rights reserved.

  This program is free software; you can redistribute it and/or modify
  it under the terms of the GNU General Public License as published by
  the Free Software Foundation; version 2 of the License.

  This program is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.

  You should have received a copy of the GNU General Public License
  along with this program; if not, write to the Free Software
  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
</copyright>

<description>
This file lists all of the available character sets.
To make maintaining easier please:
 - keep records sorted by collation number.
 - change charsets.max-id when adding a new collation.
</description>
........
<charset name="latin1">
  <family>Western</family>
  <description>cp1252 West European</description>
  <alias>csisolatin1</alias>
  <alias>iso-8859-1</alias>
  <alias>iso-ir-100</alias>
  <alias>iso_8859-1</alias>
  <alias>iso_8859-1:1987</alias>
  <alias>l1</alias>
  <alias>latin1</alias>
  <collation name="latin1_german1_ci"   id="5"  order="German Duden"/>
  <collation name="latin1_swedish_ci"   id="8"  order="Finnish, Swedish">
    <flag>primary</flag>
    <flag>compiled</flag>
  </collation>
  <collation name="latin1_danish_ci"    id="15" order="Danish"/>
  <collation name="latin1_german2_ci"   id="31" order="German Phonebook"        flag="compiled"/>
  <collation name="latin1_spanish_ci"   id="94" order="Spanish"/>
  <collation name="latin1_bin"          id="47" order="Binary">
    <flag>binary</flag>
    <flag>compiled</flag>
  </collation>
  <collation name="latin1_general_ci"   id="48">
    <order>Dutch</order>
    <order>English</order>
    <order>French</order>
    <order>German Duden</order>
    <order>Italian</order>
    <order>Latin</order>
    <order>Portuguese</order>
    <order>Spanish</order>
  </collation>
  <collation name="latin1_general_cs"   id="49">
    <order>Dutch</order>
    <order>English</order>
    <order>French</order>
    <order>German Duden</order>
    <order>Italian</order>
    <order>Latin</order>
    <order>Portuguese</order>
    <order>Spanish</order>
  </collation>
</charset>
....
</charsets>

Anyone ever experienced this particular error?

Need suggestions, thank you!

Answer 1

You are running a newer master and an older slave. That is not a supported configuration.

http://dev.mysql.com/doc/refman/5.5/en/replication-compatibility.html

I suspect this error is not actually about latin1_swedish_ci, but is actually about utf8mb4_general_ci, which may have been active on the connection on the master where the query that's failing to replicate was initially issued, so the slave is trying to execute it in that context.

I don't think this capability (utf8mb4) exists in MySQL 5.1, as evidenced by the query below, making this yet another reason why you never run a version on a slave that is lower than the version on the master, and when doing upgrades, you always want to upgrade the slave first.

SELECT * FROM information_schema.collations WHERE id = 45 returns a value when I run it on MySQL 5.5 and 5.6 but not on 5.1.

Upgrading the slave is the most correct solution, but in a pinch, you can connect directly to the slave, issue the exact same query yourself, and if it works, you could then direct the slave to skip over that event in the relay log, since you've already executed that query yourself. After executing the failing query:

SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;
START SLAVE SQL_THREAD;

This is a delicate operation, applicable only when You Know Exactly What You're Doing™ and it's likely you're going to see subsequent errors due to this same issue, so not likely to be of much benefit unless this is an isolated incident.

On the bright side, even though it's not ideal to upgrade the slave while it's stalled on a replication error, the odds of it actually succeeding and you being able to continue replication from the point where it is stopped are pretty good.