How do I safely decode a degrees symbol in a wxPython app?

Question

I have a debug app I've been writing which receives data from a C-based process via UDP. One of the strings sent to me contains a ° character - Unicode U+00B0 (which incidentally breaks the StackOverflow search function!). When my wxPython application tries to append that string to a text box I get a UnicodeDecodeError.

My first attempt to fix the issue simply caught that error (because the app apparently does send some bad messages. The problem is that the app also uses the character to report various temperatures around the unit and that's something we really need to log. Changing the source app is out of my control, so how can I detect and decode those symbols into something the wxTextCtrl can display?

Answer 1

pdc got it right, the following works fine (but fails without the decode):

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import wx

app = wx.PySimpleApp()
app.TopWindow = wx.Frame(None)
field = wx.TextCtrl(app.TopWindow)
field.Value += '°'.decode('ISO8859-1')
app.TopWindow.Show()
app.MainLoop()

Answer 2

I can't say mych about wxPython itself, but I am guessing that it is trying to convert the text to Unicode before displaying it, If you have a string like '123\xB0' and try to convert it to Unicode with teh default encoding (ASCII) then it will throw UnicodeDecodeError. You can probably fix this by replacing

s = message.get_string()

with

s = message.get_string().decode('ISO8859-1')

(where I am assuming a function get_string() that gets the message as a string). The difference here is that by handong the conversion to Unicode yourself you get to specify the encoding.

Answer 3

Things may have been different back when this was asked, but my thoughts for anyone who stumbles on this:

The issue is wxPython is trying to convert TO unicode, and lacking charset information it tries to use ASCII, which is invalid. If you know your data is utf-8, tell it so and it'll just work.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import wx

app = wx.PySimpleApp()
app.TopWindow = wx.Frame(None)
field = wx.TextCtrl(app.TopWindow)

string_data = '°'
print type(string_data)
# this would error, as it tries to convert to unicode from ascii
# field.Value += string_data

unicode_data = unicode(string_data, 'utf-8')
print type(unicode_data)
field.Value += unicode_data
app.TopWindow.Show()
app.MainLoop()