Convert mysql query from latin1 to utf8 using nodejs

Question

I have a database in latin1 and I want to convert its queries to utf-8.

Example:

response.write returns this JSON:

{"questions":[{"question":"Nação","answer":"País"}]}

But I need it converted to utf8:

{"questions":[{"question":"Nação","answer":"País"}]}




conexao_bd.getConnection(function(err,con){
            if(err) return console.log("Erro na conexao. "+err);
            con.query("SELECT question, answer FROM "+tipo+" WHERE id IN "+c,function(err, rows){
                if(err) console.log("Erro na query questions: "+err);
                else{
                    for(var k =0;k<rows.length;k++)
                        perguntas.questions[k]={'question': rows[k].question,'answer':rows[k].answer};
                    console.log(JSON.stringify(perguntas));
                    con.release();
                    response.writeHeader(200, {"Content-Type": "text/plain"});
                    response.write(JSON.stringify(perguntas));
                    response.end();
                }
            });

        });

Answer 1

Re-coding strings as posted by @bodokaiser is also a possibility, but it is not a clean system solution. The clean solution is to tell the database to give you utf-8. This can be done in two ways:

1. Set the connection charset using a query set names utf8.
2. Set the connection charset when you create connection. Don't know if this is possible in node.js. If not use the above option.

Answer 2

You will need an extra module for converting different string encodings. node-iconv seems to be one you are looking for! It is also mentained by the core developer of libuv/node so it should be quite sustainable. However think about updating the encoding on the database if possible as this may be the most lasting solution.

PS: With node-mysql it also should be possible to directly pipe the incoming data to a encoding stream (will probably not work but you get an idea):

var mysql = require('mysql');
var iconv = require('iconv');

var connection = mysql.createConnection({
  host     : 'localhost',
  user     : 'me',
  password : 'secret'
});

connection.connect();
var query = connection.query('SELECT 1 + 1 AS solution')
var encoder = new iconv.Iconv('latin1', 'utf-8');

// will encode result and pipe it to your terminal
query.pipe(encoder.pipe(process.stdout));

EDIT: Thomas made a good suggestion about setting the charset in node-mysql if updating the database is not an options. This also seems possible see here and should better than my hack. However if you are not using node-mysql this may be a workaround.