How to pass Unicode characters as JSP/Servlet request.getParameter?

Question

After a lot of trial and error I still can't figure out the problem. The JSP, servlet, and database are all set to accept UTF-8 encoding, but even still whenever I use request.getParameter on anything that has any two-byte characters like the em dash they get scrambled up as broken characters.

I've made manual submissions to the database and it's able to accept these characters, no problem. And if I pull the text from the database in a servlet and print it in my jsp page's form it displays no problem.

The only time I've found that it comes back as broken characters is when I try and display it elsewhere after retrieving it using request.getParameter.

Has anyone else had this problem? How can I fix it?

Answer 1

That can happen if request and/or response encoding isn't properly set at all.

For GET requests, you need to configure it at the servletcontainer level. It's unclear which one you're using, but for in example Tomcat that's to be done by URIEncoding attribute in <Connector> element in its /conf/server.xml.

<Connector ... URIEncoding="UTF-8">

For POST requests, you need to create a filter which is mapped on the desired URL pattern covering all those POST requests. E.g. *.jsp or even /*. Do the following job in doFilter():

request.setCharacterEncoding("UTF-8");
chain.doFilter(request, response);

For HTML responses and client side encoding of submitted HTML form input values, you need to set the JSP page encoding. Add this to top of the JSP (you've probably already done it properly given the fact that displaying UTF-8 straight form DB works fine).

<%@page pageEncoding="UTF-8" %>

Or to prevent copypasting this over every single JSP, configure it once in web.xml:

<jsp-config>
    <jsp-property-group>
        <url-pattern>*.jsp</url-pattern>
        <page-encoding>UTF-8</page-encoding>
    </jsp-property-group>
</jsp-config>

For source code files and stdout (IDE console), you need to set the IDE workspace encoding. It's unclear which one you're using, but for in example Eclipse that's to be done by setting Window > Preferences > General > Workspace > Text File Encoding to UTF-8.

Do note that HTML <meta http-equiv> tags are ignored when page is served over HTTP. It's only considered when page is opened from local disk file system via file://. Also specifying <form accept-charset> is unnecessary as it already defaults to response encoding used during serving the HTML page with the form. See also W3 HTML specification.

See also:

• Unicode - How to get the characters right?
• Why does POST not honor charset, but an AJAX request does? tomcat 6
• HTML : Form does not send UTF-8 format inputs
• Unicode characters in servlet application are shown as question marks
• Bad UTF-8 encoding when writing to database (reading is OK)

Answer 2

BalusC's answer is correct but I just want to add it is important (for POST method of course) that

request.setCharacterEncoding("UTF-8");

is called before you read any parameter. This is how reading parameter is implemented:

@Override
public String getParameter(String name) {
    if (!parametersParsed) {
        parseParameters();
    }
    return coyoteRequest.getParameters().getParameter(name);
}

As you can see there is a flag parametersParsed that is set when you read any parameter for the first time, parseParameters() method with parse all the request's parameters and set the encoding. Calling:

request.setCharacterEncoding("UTF-8");

after the parameters were parsed will have no effect! That is why some people are complaining that setting the request's encoding is not working. Most answers here suggest to use servlet filter and set the character encoding there. This is correct but also be aware that some security libraries can read request parameters before your filter (this was my case) so if your filter is executed after that the character encoding of request parameters are already set and setting UTF-8 or any other will have no effect.

Answer 3

The Tomcat FAQ covers this topic pretty well. Particularly: http://wiki.apache.org/tomcat/FAQ/CharacterEncoding#Q8 and http://wiki.apache.org/tomcat/FAQ/CharacterEncoding#Q4

The test JSP given in the FAQ is essentially the one I used when going through Tomcat years ago fixing various encoding issues.

Answer 4

Just want to add a point that in case anyone else made the same mistake as me where i overlooked POST method

Read all these solutions and applied to my code but it still didnt work because i forgot to add method="POST" in my <form> tag