quopri decoding on app engine blobstore upload

Question

I want to save a ★ (star) character to the app engine datastore but i'm having encoding / decoding issues.

• Unfortunately It works on the dev server but not on production.
• on previous apps quopri.decodestring has solved my problems

Firstly I ajax the form from AngularJS (should be irrelevant):

$http({
       method: 'POST',
       url: the_upload_url,
       transformRequest: formDataObject,
       data: event,
       headers: {'Content-Type': false}
       })

I have a webapp2.RequestHandler & blobstore_handlers.BlobstoreUploadHandler:

class EditEventHandler(ApiHandler, blobstore_handlers.BlobstoreUploadHandler):
    def post(self):
         ...
         print self.request.get('message', 'none')
         print len(self.request.get('message', 'none'))
         message = unicode(quopri.decodestring(self.request.get('message', 'none')), 'iso_8859-2')
         print message
         print len(message)
         ...
         #save message to datastore in ndb.TextProperty & send response

Here is the log output from the app engine dashboard:

2013-11-01 10:23:52.506 /admin/event/edit 200 168ms 0kb Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36
0.1.0.30 - - [31/Oct/2013:14:23:52 -0700] "POST /admin/event/edit HTTP/1.1" 200 159 "http://www.thebedford.co.nz/dashboard.html" "Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36" "www.thebedford.co.nz" ms=168 cpu_ms=53 cpm_usd=0.000018 app_engine_release=1.8.6 instance=00c61b117c6716cf4503e5f8bfe28fc9e9acd94a
E 2013-11-01 10:23:52.504
4piF
E 2013-11-01 10:23:52.504
4
E 2013-11-01 10:23:52.504
4piF
E 2013-11-01 10:23:52.504
4

As you can see the star is converted to 4piF. Where am I going wrong?

Below is the POST request in case that's helpful:

Request URL:http://www.thebedford.co.nz/_ah/upload/AMmfu6bKvvqvW6BaQGoGAHTPxBRJe6RafpzaET4pbBxXy-B9TELRochhlRBsZfNGd90xNROxF-dDE4iupO1pmcvey6wUYt46irOFBYyjmmAB_UlVWV8zJ0Lx192jMuSlESRA7dhrbD0k/ALBNUaYAAAAAUnLMvozhzPJMoRmK7kL76D3OWtuJ-dDb/
Request Method:POST
Status Code:200 OK
Request Headersview source
Accept:application/json, text/plain, */*
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Cache-Control:no-cache
Connection:keep-alive
Content-Length:2867
Content-Type:multipart/form-data; boundary=----WebKitFormBoundaryBBenbmUrxmLeoIgJ
Host:www.thebedford.co.nz
Origin:http://www.thebedford.co.nz
Pragma:no-cache
Referer:http://www.thebedford.co.nz/dashboard.html
User-Agent:Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36
Request Payload
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="img_slider"

http://lh6.ggpht.com/qC8JR3gNgi9hvuEzkBirWc0AiBMfwhZRz6PMNsZsHefo2JtR-X4vzOvJ-NvwWtWJ4EMquujsTyvh9TgPECqslFA
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="on_homepage"

true
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="event_ticket_link"

http://www.dashtickets.co.nz/event/c2g98fn74
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="date_minute"

0
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="message"

★
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="on_slider"

true
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="img"

http://lh4.ggpht.com/ivvwATwxPL6w8MFb1ox1TABE7M0MIAk-qY4ak69DUn8GrDCEYvDsGFGWxBYPgTNSsUiop5jKYz24gMtmRR6X9vU
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="formatted_date"

Sat 02 Nov  2013
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="date_day"

2
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="date_hour"

22
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="description"

4piF4piF4piF4piF
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="event_name"

Aotearoa Bass ALLSTARZ + NZ Tour +
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="date_year"

2013
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="start_time"

10:00 PM
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="venuemap"

map1
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="key"

ag1zfmJlZGZvcmQtYXBwchcLEgpFdmVudE1vZGVsGICAgICA4pQKDA
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="phone"

4piF
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="date_month"

11
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="notes"


------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="venue"

126 Madras Street, CPSA Building Christchurch, New Zealand 8011
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="end_time"

Ti9B4piF
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="on_eventguide"

true
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="event_image"

undefined
------WebKitFormBoundaryBBenbmUrxmLeoIgJ
Content-Disposition: form-data; name="event_slider_image"

undefined
------WebKitFormBoundaryBBenbmUrxmLeoIgJ--
Response Headersview source
Alternate-Protocol:80:quic,80:quic
Cache-Control:no-cache, must-revalidate
Connection:Keep-Alive
Content-Encoding:gzip
Content-Length:74
Content-Type:text/html; charset=utf-8
Date:Thu, 31 Oct 2013 21:23:52 GMT
Expires:Fri, 01 Jan 1990 00:00:00 GMT
Pragma:no-cache
Server:HTTP Upload Server Built on Oct 23 2013 16:50:37 (1382572237)
Vary:Accept-Encoding
X-AppEngine-Estimated-CPM-US-Dollars:$0.000018
X-AppEngine-Resource-Usage:ms=168 cpu_ms=64

Answer 1

The problem is that if you don't specify the version of webob, appengine uses the old webob for compatibility (I suppose the dev server uses the new webob by default hence the discrepancies). However the old version of webob ignores encodings which is a bug.

For me this was simple to fix by adding the new webob:

libraries:
- name: webob
  version: "1.2.3"

There is also a monkey patch if you don't want to upgrade your webob version.