iOS: ZBar SDK unicode characters

Question

When scanning QR codes with ZBar the string resulting from the process does not display unicode characters properly. The word Márti encoded as a QR code by any free to use QR code generator (like http://qrcode.kaywa.com) would result in Mﾃ｡rti.

In other SO questions (1, 2) it was suggested to embed a BOM at the start of the resulting string, but doing this:

NSString *qrString = [NSString stringWithFormat:@"\xEF\xBB\xBF%@",symbol.data];

or this:

NSString *qrString = [[NSString alloc] initWithFormat:@"\357\273\277%@", symbol.data];

resulted in the same, flawed result with the Asian character. symbol.data is the resulting NSString provided by ZBar.

UPDATE: Based on dda's answer, the solution was the following:

NSString *qrString = symbol.data;
//look for misinterpreted acute characters and convert them to UTF-8
if ([qrString canBeConvertedToEncoding:NSShiftJISStringEncoding]) {
  qrString = [NSString stringWithCString:[symbol.data cStringUsingEncoding: NSShiftJISStringEncoding] encoding:NSUTF8StringEncoding];
}

Answer 1

I could create QR codes of "日本語"(japanese) and "Márti" with following libraries:

• iOS-QR-Code-Encoder
• QR-Code-Encoder-for-Objective-C.

You can read those QR codes with ZBar.

iOS-QR-Code-Encoder:

NSString* orginalString = @"Márti"(or "日本語"(japanese));  
NSString *data = [[NSString alloc] initWithFormat:@"\357\273\277%@", orginalString];  
UIImage* qrcodeImage = [QRCodeGenerator qrImageForString:data imageSize:imageView.bounds.size.width];

QR-Code-Encoder-for-Objective-C:

NSString* orginalString = @"Márti"(or "日本語"(japanese));
NSString *data = [[NSString alloc] initWithFormat:@"\357\273\277%@", orginalString];

//first encode the string into a matrix of bools, TRUE for black dot and FALSE for white. Let the encoder decide the error correction level and version
DataMatrix* qrMatrix = [QREncoder encodeWithECLevel:QR_ECLEVEL_AUTO version:QR_VERSION_AUTO string:data];

//then render the matrix
UIImage* qrcodeImage = [QREncoder renderDataMatrix:qrMatrix imageDimension:qrcodeImageDimension];

Answer 2

According to the Wikipedia page about QR, the encoding of binary data [for which Márti would apply] is ISO 8859-1. It could be an encoding-as-unicode-encoding problem. But seeing a kanji there, it could be that the problem is an encoding-as-QR-encoding issue: maybe the text, being not ASCII, is encoded by default as Shift JIS X 0208 (ie kanji/kana).

Answer 3

Just a word of caution, the solution as is will exclude use in Japan and scanning of QR Codes with actual Kanji coding inside. In fact it will probably create problems for any QR Code with Unicode characters inside that canBeConvertedToEncoding:NSShiftJISStringEncoding.

A more universal solution is to insert the BOM characters prior to the QR Code encoding to force UTF-8 coding (before it is created). ZBar was never the problem here, it is rooted in the creation of the QR Code.