Getting "OperationalError: 1366" from python script - MySQL 5.6 . MySQLdb 1.2.3

Question

Environment:

• MySQL-python-1.2.3.win-amd64-py2.7

• python-2.7.amd64

• mysql-installer-community-5.6.12.2

• Windows Server 2008 R2 Datacenter

I am getting Operational Error 1366 when inserting data with a python script. I need to run the EPFImporter which is from 2010

I think the problem is that MySQLdb can only handle uft8 but not utf8mb4_unicode_ci. The data I need to import contains utf8mb4 charackters and I may not remove them.

Running the script from CMD comand line gives me following error message after it succesfully inserted ~70.000 rows:

2013-07-31 15:14:53,460 [ERROR]: Fatal error encountered while ingesting 'C:\webserver\www\site1\assets\scripts
Traceback (most recent call last):
  File "C:\webserver\www\site1\assets\scripts\EPFImporter\EPFIngester.py", line 129, in ingestFull
    self._populateTable(self.tmpTableName, skipKeyViolators=skipKeyViolators)
  File "C:\webserver\www\site1\assets\scripts\EPFImporter\EPFIngester.py", line 379, in _populateTable
    cur.execute(exStr)
  File "C:\Windows\python\lib\site-packages\MySQLdb\cursors.py", line 174, in execute
    self.errorhandler(self, exc, value)
  File "C:\Windows\python\lib\site-packages\MySQLdb\connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x98\\x8A A...' for column 'description' at row 1")

I allready made:

ALTER DATABASE databaseXYZ CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci

And I changed following from INNODB to:

default storage engine  MyISAM  Documentation
default tmp storage engine  MyISAM 

before this change it was not importing any row at all.

What else can I do? Generally it should be working since on a shared host I was running it on MySQL Server Version: 5.5.28 and it worked fine

EDIT1: For testing I installed the latest XAMPP version and under XAMPP the script is running fine with only having the XAMPP-MySQL Server running not the Apache. Python and MySQLdb versions remained the same as before and were untouched. So I am now pretty sure that i did something wrong in the configuration of the MySQL-Server. In XAMPP the DB is running with InnoDB Engine.

Here are the speccs of the XAMPP MYSQL Server:

Server: 127.0.0.1 via TCP/IP
Server type: MySQL
Server version: 5.6.11 - MySQL Community Server (GPL)
Protocol version: 10
User: root@localhost
Server charset: UTF-8 Unicode (utf8)

Does anyone has an idea what I could do to trace down the problem in my original MySQL setup? Could it be something in the variable setting, the MYSQL error handling or the Python connector?

Answer 1

I solved the problem by comparing the XAMPP-MySQL and regular MySQL values in phpmyadmin > localhost > variables

I changed a few values in the regular MySQL variables so I am not 100% sure which change finally solved the issue but I guess it was changing to:

sql mode    NO_ENGINE_SUBSTITUTION

I had a few more other values like strict_mode in there before which i deleted.