SQL Service Broker Queue Handling Foreign Characters in the message

Question

Setup: I have a form built in asp.net/c# that, on submit, XML serializes it's object model and calls a stored procedure with that XML serialized data as the sole parameter. The stored procedure sends that data to a sql broker queue. The message sent to the broker queue must be valid XML that obeys the message contract set on the queue. That message is picked up by BizTalk and processed accordingly.

Problem: Originally the data submitted to me was just regular English characters (essentially held to ASCII charset) but a requirement is on the horizon to support foreign characters as well. In my testing, I've noticed that if I try to submit something with foreign characters (chinese, arabic, etc), I get an error in the queue and the message that gets to BizTalk ends up with "?????" in place of the foreign characters. I've added the utf=16 xml header to the top of the document, but that doesn't seem to help.

Question: Is there a way I can cast the incoming XML message as nvarchar and still have it be considered valid XML by the queue? I don't want to change the actual type on the queue or recreate it. I'd prefer to change the message in the stored proc alone in some way that allows it to get on the queue.

Thanks in advance for your help.

Answer 1

I ended up handling this by encoding the characters using HTML5 and then security escaping them. I ran into some issues using the HttpUtility library to handle this encoding so I added the method that I used to handle the encoding.

I wish I could give direct credit for this, I can't remember where I found this but thank you to whomever it was:

private string EncodeToHTML(string text) { // call the normal HtmlEncode first char[] chars = HttpUtility.HtmlEncode(text).ToCharArray(); StringBuilder encodedValue = new StringBuilder(); foreach (char c in chars) { if ((int)c > 127) // above normal ASCII encodedValue.Append("&#" + (int)c + ";"); else encodedValue.Append(c); } return encodedValue.ToString(); }