how to deal with special characters in nodejs fs readdir function

Question

I'm reading a directory in nodejs using the fs.readdir() function. You feed it a string containing a path and it returns an array containing all the files inside that directory path in string format. It does not work for me with special characters (like ï).

I came across this similar issue, however I am on OS X).

First I created a new dir called encoding and created a file called maïs.md (with my editor Sublime Text).

fs.readdir('encoding', function(err, files) {
  console.log(files);                                   // [ 'maïs.md' ]
  console.log(files[0]);                                // maïs.md
  console.log(files[0] === 'maïs.md');                  // false
  console.log(files[0] == 'maïs.md');                   // false
  console.log(files[0].toString('utf8') === 'maïs.md'); // false
});

The above test works correctly for files without special characters. How can I compare this correctly?

Answer 1

you character seems to be this one. You should try with

(1) console.log(files[0] == 'ma\u00EF;s.md'); 
(2) console.log(files[0] == 'mai\u0308;s.md'); 

If (1) works it could mean that the file containing your code is not saved in utf-8 format, so the node.js engine does not interpret correctly the ï character in your code.

If (2) works it could mean that the file system gives to the node engine the ï character in its decomposed unicode form (i followed by a diacritic ¨). cf @thejh answer

In this (2) case, use the unorm library available on npm to normalize the strings before comparing them (or the original UnicodeNormalizer)

Answer 2

https://apple.stackexchange.com/a/10484/23863 looks relevant – it's probably because there are different ways to express ï in utf8.