NSTask string encoding problem

Question

In my program, I'm grep-ing via NSTask. For some reason, sometimes I would get no results (even though the code was apparently the same as the command run from the CLI which worked just fine), so I checked through my code and found, in Apple's documentation, that when adding arguments to an NSTask object, "the NSTask object converts both path and the strings in arguments to appropriate C-style strings (using fileSystemRepresentation) before passing them to the task via argv[]" (snip).

The problem is that I might grep terms like "Río Gallegos". Sadly (as I checked with fileSystemRepresentation), that undergoes the conversion and turns out to be "RiÃÅo Gallegos".

How can I solve this?

-- Ry

Answer 1

The problem is that I might grep terms like "Río Gallegos". Sadly (as I checked with fileSystemRepresentation), that undergoes the conversion and turns out to be "RiÃÅo Gallegos".

That's one possible interpretation. What you mean is that “Río Gallegos” gets converted to “Ri\xcc\x81o Gallegos”—the UTF-8 bytes to represent the decomposed i + combining acute accent.

Your problem is that grep is not interpreting these bytes as UTF-8. grep is using some other encoding—apparently, MacRoman.

The solution is to tell grep to use UTF-8. That requires setting the LC_ALL variable in your grep task's environment.

The quick and dirty value to use would be “en_US.UTF-8”; a more proper way would be to get the language code for the user's primary preferred language, replace the hyphen, if any, with an underscore, and stick “.UTF-8” on the end of that.