Question

I'm looking for sample 1-byte, 2-byte, 3-byte, 4-byte, 5-byte, and 6-byte unicode characters. Any links to some sort of reference of all the different unicode characters out there and how big they are (byte-wise) would be greatly appreciated. I'm hoping this reference also has code points like \uXXXXX.

Was it helpful?

OTHER TIPS

There is no such thing as "1-byte, 2-byte, 3-byte, 4-byte, 5-byte, and 6-byte unicode characters".

You probably talk about UTF-8 representations of Unicode characters. Similarly, strings in Java are internally represented in UTF-16, so that Java char type represents a 16-bit code unit of UTF-16, and each Unicode character can be represented by either one or two these code units, and each code unit can be represented as \uxxxx in string literals (note that there are only 4 hex digits in these sequences, since code units are 16-bit long).

So, if you need a reference of Unicode characters with their UTF-8 and UTF-16 representations, you can take a look at the table at fileformat.info.

See also:

As axtavt points out, the concept of n-byte Unicode characters is meaningless; assuming you mean UTF-8, then a very simple table, which might help you with testing etc, might be as follows. Note that all example characters work on my browser (Chrome on Ubuntu) but your mileage may vary in terms of displaying, copying/pasting, etc.

UTF-8 bytes  Start    End       Example Character
1            U+0000   U+007F    ! EXCLAMATION MARK U+0021)
2            U+0080   U+07FF    ¶ PILCROW SIGN (U+00B6)
3            U+0800   U+FFFF    ‱ PER TEN THOUSAND SIGN (U+2031)
4            U+10000  U+1FFFFF  𝅘𝅥𝅯 MUSICAL SYMBOL SIXTEENTH NOTE (U+1D161)

In theory there can be 5- or 6- byte values in UTF-8, but Unicode's 32-bit address space is limited in reality to a max of 10FFFF so more than 4 bytes aren't required.

Note that there's an important caveat here: Java's char is not a Unicode character; it's a 16-bit code unit of UTF-16, and it is not uncommon to see data streams which treat a non-BMP character (like U+1D161 above) as 2 characters, and UTF-8 it accordingly. For example:

Character: U+1D161
UTF-8 encoding: 0xF0 0x9D 0x85 0xA1
UTF-16 encoding: 0xD834 0xDD61
UTF-16 code points individually encoded as UTF-8: 0xED 0xA0 0xB4 0xED 0xB5 0xA1

Note that this has the effect of apparently showing a 6-byte UTF-8 character, but this is in fact not permitted by UTF-8. UTF-8 must be the encoding of the original code points, not the encoding of the UTF-16 code units which represents those points. This doesn't mean you don't see it in the wild though...

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top