Java UTF-8 Encoding Issue

Question

I wonder if anyone can please help, I am having difficulty posting UTF-8 characters to SagePay. The database is MySQL with Database charset utf8 and Database collation utf8_general_ci. The database connection string uses useUnicode=true&characterEncoding=UTF-8 and the jsp page is correctly encoded as <%@page pageEncoding="UTF-8" contentType="text/html; charset=UTF-8"%>

All data posted to database is stored as UTF-8, all data queried is presented as UTF-8 but when a string is encoded as follows:

crypt = Base64Coder.encodeString(encXor(strPost));

and then posted to SagePay, they receive a garbled string from the point of wherever a international character is found. If no international characters are contained within the string the post to SapePay is successful.

My questions are if data queried is UTF-8 why is the encryption or encoding prior to post not UTF-8 and also how can I force UTF-8 encoding or possible force ISO-8859-1 to post to SagePay, Granted I'd prefer to keep UTF-8 but struggling to find a solution.

The Xor function is as follows:

public static String encXor(String s)
{
    String s1 = "password";
    String s2 = null;
    byte abyte0[] = s.getBytes();
    byte abyte1[] = s1.getBytes();
    int i = 0;
    int j = abyte1.length;
    ByteArrayOutputStream bytearrayoutputstream = new ByteArrayOutputStream(abyte0.length);
    for(int k = 0; k < abyte0.length; k++)
    {
        byte byte0 = abyte0[k];
        byte byte1 = abyte1[i];
        byte byte2 = (byte)(byte0 ^ byte1);
        if(i < j - 1)
            i++;
        else
            i = 0;
        bytearrayoutputstream.write(byte2);
    }

    try
    {
        bytearrayoutputstream.flush();
        s2 = bytearrayoutputstream.toString();
        bytearrayoutputstream.close();
        bytearrayoutputstream = null;
    }
    catch(IOException ioexception) { }
    return s2;
}

Any help would be much appreciated :-)

Answer 1

I discovered the issue, it did not relate to the code but actually some start-up parameters. Using Tomcat7w.exe I noticed in the Java tab, the following lines:

-Dfile.encoding=UTF-8
-Dsun.jnu.encoding=UTF-8

I removed these lines, restarted the Tomcat service and everything worked, therefore no code alterations were required.

Thank you to all that responded :-)

Answer 2

When you do

byte abyte0[] = s.getBytes();
byte abyte1[] = s1.getBytes();

you use the Java platform encoding, which might not be UTF-8. And what is worse, it may be different between your client and server.

Try to specify the encoding explicitly:

byte abyte0[] = s.getBytes(StandardCharsets.UTF_8);
byte abyte1[] = s1.getBytes(StandardCharsets.UTF_8);

The same applies, when you convert back from bytes to a string.