MySQL to PHP to NSData to NSString - Where has my UTF8 encoding gone wrong?

Question

I'm trying to write a simple PHP web service as a bridge between a MySQL database on my web server and an Objective-C Cocoa desktop app. Every link in the chain is working in UTF-8, as best as I can tell, but I can't seem to get the non-standard characters to display in Cocoa; instead, my interface elements show only the escapes.

I'm open to any solution that lets me fix this problem at any stage; I'm writing the PHP and the Obj-C side, so I should be able to deal with problems on the server side or on the client side. I just don't know which mole to whack at this point.

The Database

The information is coming from a table in a MySQL database; the database collation and each field are set to utf_general_ci.

PDO

My PHP web service fetches the data using PDO:

    $pdo = new PDO('mysql:host=mysql.hostname.com;dbname=tst_data_model', 'user', 'password', array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8"));
if ($pdo) {} else {die();}

$stmt = $pdo->prepare("SELECT * FROM team");
$stmt->execute();

PHP

I then echo the data:

header('Content-Type: text/html; charset=utf-8'); 
echo("<TEAM>\r");
echo("<team_id>{$row['team_id']}</team_id>\r");
echo("<name_full_native>{$row['name_full_native']}</name_full_native>\r");
echo("<name_short_native>{$row['name_short_native']}</name_short_native>\r");
echo("<name_full_english>{$row['name_full_english']}</name_full_english>\r");
echo("<name_short_english>{$row['name_short_english']}</name_short_english>\r");
echo("</TEAM>\r\r");

NSDATA

The desktop app fetches this data using NSURLConnection:

NSData* returnValue = [NSURLConnection sendSynchronousRequest:teamArrayRequest returningResponse:NULL error:NULL];

NSString

Finally, I pull the string out of the data, specifying UTF8 string encoding:

NSString* rVString = [[NSString alloc] initWithData:returnValue encoding:NSUTF8StringEncoding];

When I view this string in the debugger, or send it to an NSTextView, all I see are \u escapes:

Borussia M\U00f6nchengladbach

Quer\U00e9taro

Real Espa\U00f1a

Am I missing something? Where are these escapes coming from?

Answer 1

Uh...never mind. Turned out this was a case of PEBCAK.

In fact, everything was in UTF8 after all, all the way to the NSString. What confused me is that what I was viewing in the debugger, and passing to the NSTextView, wasn't the NSString; it was the output of the -description method of either an NSArray or an NSDictionary containing the string that I was immediately assigning it to after I'd fetched it.

Turns out the -description method of either NSArray or NSDictionary always, for some reason, outputs UTF-8 characters other than standard ASCII characters as /u escapes.

When I actually pulled the string out of the array of dictionaries I was using to parse and store it and plugged the string directly into the NSTextView, it displayed as perfect UTF8:

Borussia Mönchengladbach

Querétaro

Real España

So...that's the answer. [NSArray description] and [NSDictionary description] do not display every character in a string value as it will display as an NSString. I guess this is a choice the coders made so that whitespace characters, etc. don't show up in plist descriptions.

Although I know I've seen Chinese characters in plist descriptions and debugger output before...so it's still a little mysterious. Still, problem solved. Sorry to waste the hive-mind's time, and I hope the explanation helps someone down the road.