mysql in python encoding

Question

This post is the same with my question in MySQL in Python: UnicodeEncodeError: 'ascii' this is just to clear things up.

I am trying to save a string to a MySQL database but I get an error:

File ".smart.py", line 51, in (number, text, 'smart', 'u')

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 25: ordinal not in range(128)

and the string is saved at m['Text']

Lala*=#&%@<>_?!:;-'"/()¥¡¿

Here is a snippet to the code

risk = m['Text']
msg = risk.encode('utf8')
text = db.escape_string(msg)

sql = "INSERT INTO posts(nmbr, \
       msg, tel, sts) \
       VALUES ('%s', '%s', '%s', '%s')" % \
       (number, text, 'smart', 'u')

If i try to comment out the SQL query and put print text it would print out Lala*=#&%@<>_?!:;-'"/()¥¡¿

The error is only encountered when the SQL is being processed.

MySQL encoding is set to utf8_unicode_ci. (or should i change this?)

Thanks.

Answer 1

add these parameters MySQLdb.connect(..., use_unicode=1,charset="utf8").

create a cursor

cur = db.cursor()

and then execute like so:

risk = m['Text']
sql = """INSERT INTO posts(nmbr, msg, tel, sts) \
         VALUES (%s, %s, %s, %s)"""
values = (number, risk, 'smart', 'u')
cur.execute(sql,values)  #use comma to separate sql and values, this will ensure values are escaped/sanitized
cur.commit()

now you dont need these two lines:

msg = risk.encode('utf8')
text = db.escape_string(msg)

Answer 2

It is not clear whether your m['Text'] value is of type StringType or UnicodeType. My bet is that it is a byte-string (StringType). If that's true, then adding a line m['Text'] = m['Text'].decode('UTF-8') before your insert may work.