apostrophe in JSON is causing errors on server

Question

I have a string that I'm doing a JSON.stringify(str) on in Javascript. The string is L'Oreal.

However, as this variable gets passed around before the JSON.stringify happens, its value becomes

L& #39;Oreal(without the space between & and #), and the resulting JSON string that i send to the server is being recognized as potentially dangerous and I get a server side error.

My question is, how do I avoid the apostrophe getting replaced by <&#39>; before the stringify call, or an alternate way to solve this?

EDIT 1: Here's some code that is causing this, its quite basic -

for (var rowIndex = 0; rowIndex < numrows; rowIndex++)
{
     var cellValues = new Array();
     for (var cellIndex=0; cellIndex < numCols; cellIndex++)
     {
          cellValues[cellIndex] = someInputArray[cellIndex]; //One of the values that gets populated here inlcudes the word L'Oreal
     }
     rowValues[rowIndex] = cellValues; //After this assignment, rowValues[0][3] which was earlier L'Oreal becomes L&#39;Oreal
}

var jsonToSend = JSON.stringify(rowValues);

I tried to build a jsFiddle but I do not see this happening there in this very basic example which mimics the # of times the string gets assigned in the actual code.

EDIT 2: I know why this is happening now. We HTML Encode the data when it comes from the server to avoid XSS injection. While it displays alright, when I convert this data to JSON, it results in ill-formed JSON which the server recognizes as potentially dangerous and throws an exception.

Server-side code (.Net C#)- WebUtility.HtmlEncode(Data);

Still don't know what a good way to handle this might be.

Answer 1

var str = "This is a test string with L'Oreal in it.";
var regex = /'/g;
var output = JSON.stringify(str.replace(regex, "'"));
$('#test').html(output);

You can use a regular expression to sniff out the special character code and replace it back before stringifying it. NOTE: the "g" in the regex above makes the search global so it will replace any instance of "'". The above code should work. Here is a jsfiddle demonstrating. http://jsfiddle.net/pJD9X/1/

EDIT: Alternate whitelist approach. You could possibly create a whitelist of special characters and take an approach that is similar to how underscor.js actually encodes stuff. Only instead of encoding you will be decoding. NOTE: this is possibly a dangerous solution because it allows your code to decode special characters

var str = "This is a test string with L'Oreal in it and an ampersand & in it";
var whiteList = {
    "'":"'",
    "&":"&"
};
var specialCharDecoder = /'|&/g;
function htmlDecode (string) {
    return ('' + string).replace(specialCharDecoder, function (match) {
       return whiteList[match]; 
    });
}
var output = htmlDecode(str);

Answer 2

While the solution that Mike gave will work fine for isolated cases, in my situation the user could enter information with any special characters and there's no way to predict a set of special characters that will be used.

In order to over come this, I'm manually decoding each value before JSONifying it -

for (var rowIndex = 0; rowIndex < numrows; rowIndex++)
{
    var cellValues = new Array();
    for (var cellIndex=0; cellIndex < numCols; cellIndex++)
    {//FIX BELOW
        cellValues[cellIndex] = $('<div />').html(someInputArray[cellIndex]).text(); //manually decode
    }
    rowValues[rowIndex] = cellValues; 
}

var jsonToSend = JSON.stringify(rowValues);