Why do I need to escape unicode in java source files?

Question

Please note that I'm not asking how but why. And I don't know if it's a RCP specific problem or if it's something inherent to java.

My java source files are encoded in UTF-8.

If I define my literal strings like this :

    new Language("fr", "Français"),
    new Language("zh", "中文")

It works as I expect when I use the string in the application by launching it from Eclipse as an Eclipse application :

But if fails when I launch the .exe built by the "Eclipse Product Export Wizard" :

The solution I use is to escape the chars like this :

    new Language("fr", "Fran\u00e7ais"), // Français
    new Language("zh", "\u4e2d\u6587") // 中文

There is no problem in doing this (all my other strings are in properties files, only the languages names are hardcoded) but I'd like to understand.

I thought the compiler had to convert the java literal strings when building the bytecode. So why is the unicode escaping necessary ? Is it wrong to use use high range unicode chars in java source files ? What happens exactly to those chars at compilation and in what it is different from the handling of escaped chars ? Is the problem just related to RCP cache ?

Answer 1

It appears that the Eclipse Product Export Wizard is not interpreting your files as UTF-8. Perhaps you need to run Eclipse's JVM with the encoding set to UTF-8 (-Dfile.encoding=UTF8 in eclipse.ini)?

_{(Copypasta'd at OPs request)}

Answer 2

When exporting a plug-in, it gets compiled through a process separate from the normal build process within the IDE. There is a known bug that the build process (PDE.Build) disregards the text encoding used by the IDE.

The export can be made to work properly by specifying the text encoding in the build.properties file of your plugin

javacDefaultEncoding.. =UTF-8