Rails 3, mysql/mysql2 misinterpreting some retrieved strings as ASCII-8BIT

Question

This problem began as the commonly-seen “incompatible character encodings: ASCII-8BIT and UTF-8” problem, but that is not what I'm asking. Rather, I discovered that this problem was happening because certain fields of my database are being tagged as ASCII-8BIT when they're retrieved, while most are correctly shown as UTF-8.

For example, in a table with columns country and nationality, where both columns in row 16 have identical values (copied-and-pasted), I get

irb(main):003:0> c = Country.find(16)
irb(main):004:0> puts "#{c.name}, #{c.name.encoding}, #{c.name.bytes.to_a}"
�land Islands, UTF-8, [195, 133, 108, 97, 110, 100, 32, 73, 115, 108, 97, 110, 100, 115]
irb(main):005:0> puts "#{c.nationality}, #{c.nationality.encoding}, #{c.nationality.bytes.to_a}"
�land Islands, ASCII-8BIT, [195, 133, 108, 97, 110, 100, 32, 73, 115, 108, 97, 110, 100, 115]

Likewise, a simple puts name gives �land Islands while for nationality it gives "\xC3\x85land Islands" -- same bytes, different presentation.

The encoding for a given column appears to be constant regardless of whether the string has non-ascii characters, so it is not simply a problem with the string. That is, all the values in nationality are interpreted as ascii and all those in name as UTF-8.

The problem is not limited to a single table, and I have not found any pattern to which columns are mis-recognized.

Here are the settings and environment:

• Rails 3.0.0 on Windows 7 64-bit
• Database adapter: mysql2 and mysql both show same behavior
• Database.yml includes encoding: utf8
• application.rb includes config.encoding = "utf-8"
• MySQL database, table, and both columns are defined as utf8
• Both columns in MySQL are varchar, 255, allow null
• I can reproduce the problem with a fresh installation of Rails and nothing except the Country model defined, to access the database. I have not yet tried with a fresh, one-line database.

Anyone know what's going on here?

Answer 1

Same problem here! mysql2 gem, Rails 3.0.3 and "incompatible character encodings" errors

Answer 2

I found the solution, use ruby-mysql gem instead mysql or mysql2 gems.

Answer 3

I think I'm living a similar problem.

When I retrivied datas from MySQL DB rails convert a string to float:

Columns "Complemento" and "estado" are string in DB although action "show" says Complemento: 0.0 -> at DB is "apto 191"

Escola was successfully created.

Nome: Silva Braga

Endereco: Rua Dr Arnaldo

Numero: 99

Complemento: 0.0 -> DB is "apto 191"

Cidade: São Paulo

Estado: 0.0 -> DB is "MG"

Edit | Back