When asked to send in UTF-8, Mozilla Firefox sends me UTF-16 data (breaking my servlet)

StackOverflow https://stackoverflow.com/questions/20913716

Question

I'm having trouble in my web server with content coming from the Mozilla Firefox browser. I’m trying to figure out what I must do to the web page configuration to make it behave the same as the Chrome or MSIE11 browser. I have a web page that properly sets its headers:

<html>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

In a text field I can paste in “high-bit” text. My testing string of choice is

 ƉǮ…Ł

This is (capital-D with a slash), (a three-like character with a mark), an ellipsis and (capital-L with an angled slash). It is UTF-16 format.

I then submit this to the server. Here is the Mozilla request header:

POST /portal/SalesOrderServlet HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Referer: http://localhost:8080/portal/salesOrderEdit.jsp?sequence=1508667&req=r88a_3414
Content-Length: 21321
Cookie: JSESSIONID=8283746DD2158665EADD586BFC9B6250
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache

Here is the Chrome request header:

POST /portal/SalesOrderServlet HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Content-Length: 21320
Accept: application/json, text/javascript, */*; q=0.01
Origin: http://localhost:8080
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Referer: http://localhost:8080/portal/salesOrderEdit.jsp?sequence=1508667&req=r88a_3414
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Cookie: JSESSIONID=BECCA9297706D45011D67DF82498FD66

The data is sent to the server through an AJAX call, in JSON format. Seriously edited, it looks like this for both Firefox and Chrome:

jsonContent:[{"name":"formData","value":"{\"hold_ae_question\":false, [snip] \"op_notes\":\"Ð?…L\", [snip] }"}]

On the server (Tomcat, using a version of Java 5) I extract the data from the request:

HaloJsonObject ajaxCall = JSONUtilities.jsonify(request);

String strContent = ajaxCall.getString("jsonContent");

HaloJsonObject jsonContent = JSONUtilities.createHaloJsonObjectFromString(strContent);

The “HaloJsonObject” is a thin wrapper around JSONObject. The supporting functions are included here because I’ll be asked about them anyway:

public static HaloJsonObject jsonify(HttpServletRequest request) {
    Enumeration<String> enums = request.getParameterNames();
    JSONObject json = new JSONObject();

    while(enums.hasMoreElements()) {
        String paramName = enums.nextElement();
        String paramValue = request.getParameter(paramName);

        try {
            json.put(paramName, paramValue);
        } catch (JSONException e) {
            // log stuff
        }
    }

    return new HaloJsonObject(json, JSONObject.getNames(json));
}

public static HaloJsonObject createHaloJsonObjectFromString(String source) {
    return convertFormArrayToObject(HaloJsonArray.createJsonArrayFromString(source));
}

public static HaloJsonObject convertFormArrayToObject(JSONArray formDataArray) {
    HaloJsonObject json = new HaloJsonObject();

    for(int i = 0; i < formDataArray.length(); i++) {
        JSONObject jo = new JSONObject();            

        try {
             jo = formDataArray.getJSONObject(i);
             json.put(jo.getString("name"), jo.getString("value"));
        } catch (JSONException e) {
            // log stuff
        }
    }

    return json;
}

public static JSONArray createJsonArrayFromString(String array) {
    JSONArray json = null;

    try {
        json = new JSONArray(array);
    } catch (JSONException e) {
        json = new JSONArray();
    }

    return json;
}

Once the data is parsed into “jsonContent” the Eclipse debugger displays it this way:

When coming from Firefox it looks like:

Ð?…L

When coming from Chrome it looks like:

Æ\u0089Ç®â\u0080¦Å\u0081

or

Æ?Ç®â?¦Å

depending on where you tap into the value and how the toString() treats it. Either representation is that of the UTF-8 value of the string.

This is a disaster for me because my JDBC connector hates the UTF-16 format.

Why would seemingly identical calls, one from Chrome and the other from Firefox, yield the different results?

Thanks, Jerome.

Was it helpful?

Solution

Looks to me to be a misunderstanding. I don't explicitly call jQuery's $.ajaxSetup(), and the jQuery documentation says the default value of contentType for $.ajax() is "application/x-www-form-urlencoded; charset=UTF-8". However, without an explicit setting the Firefox browser has the UTF-8 clause and the Chrome browser does not.

When I explictly set a contentType using UTF-8 in the $.ajax() call I see the same behavior for both browsers, the Firefox behavior from before. Incidentally, although the data transfer is by UTF-8, in the server the data fields are populated with the UTF-16 values. From there I'll have to arrange to get the data saved into the database, probably by converting them to UTF-8.

Thanks for the looks. Jerome.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top