Special characters rendering improperly

StackOverflow https://stackoverflow.com/questions/23252648

Question

In a form field, a user has entered the following text:

Número Telefónico

Despite the text being rendered fine in the form field itself (even after refreshing), that same text (submitted and saved to the db) looks like this when output on a different page with <%= @foo.text %>:

Número Telefónico

(I think) I've ensured consistent use of UTF-8 encoding:

irb(main):014:0> "Número Telefónico".encoding
=> #<Encoding:UTF-8>

Via psql:

=> SHOW SERVER_ENCODING;
 server_encoding 
-----------------
 UTF8

And on the HTML side:

<meta http-equiv="content-type" content="text/html; charset=UTF-8" />

But still these characters do not render properly. Please let me know if there's more information I can provide.

UPDATE

It appears the text gets screwed up only when rendered through the Liquid templating engine. I'll post an issue on Liquid's Github, and will update this question with any new information (and hopefully an answer).

UPDATE 2

Here's the relevant Github issue.

Was it helpful?

Solution

The true culprit ended up being Nokogiri, not Liquid. Nokogiri is used to parse and manipulate the HTML that will eventually go through Liquid. During Nokogiri's time with the HTML, however, it was confusing the encoding (particularly when using doc.replace), and thus screwing up the special characters.

Relevant question and answer to this specific problem (I ended up using the solution from the second link below):

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top