How to keep the Chinese or other foreign language as they are instead of converting them into codes?

Question

DOMDocument seems to convert Chinese characters into codes, for instance,

你的乱发 will become ä½ çš„ä¹±å‘

How can I keep the Chinese or other foreign language as they are instead of converting them into codes?

Below is my simple test,

$dom = new DOMDocument();
$dom->loadHTML($html);

If I add this below before loadHTML(),

$html = mb_convert_encoding($html, "HTML-ENTITIES", "UTF-8"); 

I get,

你的乱发

Even though the coverted codes will be displayed as Chinese characters, 你的乱发 still are not 你的乱发 what I am after....

Answer 1

DOMDocument seems to convert Chinese characters into codes [...]. How can I keep the Chinese or other foreign language as they are instead of converting them into codes?

$dom = new DOMDocument();
$dom->loadHTML($html);

If you're using the loadHTML function to load a HTML chunk. By default DOMDocument expects that string to be in HTML's default encoding (ISO-8859-1) however most often the charset (sic!) is meta-information provided next to the string you're using and not inside. To make this more complicated, that meta-information be be even inside the string.

Anyway as you have not shared the string data of the HTML and you have not specified the encoding, it's hard to tell specifically what is going on.

I assume the HTML is UTF-8 encoded but this is not signalled within the HTML string. So the following work-around can help:

$doc = new DOMDocument();
$doc->loadHTML('<?xml encoding="UTF-8">' . $html);

// dirty fix
foreach ($doc->childNodes as $item)
    if ($item->nodeType == XML_PI_NODE)
        $doc->removeChild($item); // remove hack
$doc->encoding = 'UTF-8'; // insert proper

It injects an encoding hint on the very beginning (and removes it after the HTML has been loaded). From that point on, DOMDocument will return UTF-8 (as always).

Answer 2

I just stumbled upon this thread when searching for a solution of a similar problem, i after loading the html properly and doing some parsing with Xpath etc... my text ends up like this:

你的乱发

this display fine in the body of the HTML, but won't display properly in a style or script tag (e.g. setting chinese-fonts).

to fix this, do the reverse lauthiamkok did:

$html = mb_convert_encoding($html, "UTF-8", "HTML-ENTITIES");

if for any reason the first workaround doesn't work for you, try this conversion.

Answer 3

I'm pretty sure ä½ çš„ä¹±å‘ is actually Windows Latin 1 (not ASCII, there are no diacritics in ASCII). Somewhere along the way your UTF-8 text got saved as Windows Latin 1....