Python+mako Unicode problem

StackOverflow https://stackoverflow.com/questions/7389935

  •  29-10-2019
  •  | 
  •  

Question

I am trying to read a DB table contents and display it as a web page using mako and bottle. The table has some Unicode (utf-8) fields in it.

UnicodeDecodeError('ascii', 'MOTOROLA MILESTONE\xe2\x84\xa2 PLUS', 
                   18, 19, 'ordinal not in range(128)')

With the following stack trace:

Traceback (most recent call last):
  File "/workspace/web/controller/bottle.py", line 499, in handle
    return handler(**args)
  File "webserver/webserver.py", line 101, in download
    return html_tmpl(tmpl, **kwds)
  File "webserver/webserver.py", line 116, in html_tmpl
    return tmpl.render(**kwds)
  File "/usr/lib/python2.5/site-packages/Mako-0.3.4-py2.5.egg/mako/template.py", line 189, in render
    return runtime._render(self, self.callable_, args, data)
  File "/usr/lib/python2.5/site-packages/Mako-0.3.4-py2.5.egg/mako/runtime.py", line 403, in _render
    _render_context(template, callable_, context, *args, **_kwargs_for_callable(callable_, data))
  File "/usr/lib/python2.5/site-packages/Mako-0.3.4-py2.5.egg/mako/runtime.py", line 434, in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/usr/lib/python2.5/site-packages/Mako-0.3.4-py2.5.egg/mako/runtime.py", line 457, in _exec_template
    callable_(context, *args, **kwargs)
  File "download_android_index_html", line 41, in render_body
  File "download_android_index_html", line 23, in fill_devices
  File "download_android_index_html", line 68, in render_fill_devices
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 18: ordinal not in range(128)

The calling function is:

def html_tmpl(tmpl, **kwds):
    kwds['nav'] = templates_lookup.get_template('nav.html').render()
    kwds['nav_bottom'] = templates_lookup.get_template('nav_bottom.html').render()
    base_path = request.path.replace("de/","").replace("fr/","")
    kwds['languages'] = templates_lookup.get_template('languages.html').render(en_url=base_path,fr_url="/fr"+base_path)
    kwds['analytics'] = ''
    return tmpl.render(**kwds)

How do I go aboutthis? I've tried:

return tmpl.render_unicode(**kwds)`

and 

return tmpl.render_unicode(**kwds).encode('utf-8', 'replace')

with no luck, and this answer did not help much.

Any ideas?

Was it helpful?

Solution

The problem is not that render_unicode cannot convert a python unicode object into utf8, its that a string object exists, which it assumes is ascii, and that holds non ascii data.

Start at the beginning - decode all incoming strings into unicode internally. You have a string input that needs fixing.

I suggest you try naming all variables at the boundary with a sort of hungarian notation - perhaps rawstr_myvar and u_myvar.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top