Google App Engine SDK urlfetch workaround for openssl bug in Ubuntu 12.04

Question

Ubuntu 12.04 has an open bug regarding openssl. If you try to connect to a host that does not support TLS 1.2 (for example Evernote), you get "EOF occurred in violation of protocol". See https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/965371

In the comments to that bug, there is a workaround for httplib.py, where TLS 1.1 is forced. Unfortunately this does not work for the GAE SDK, as it uses its own custom httplib based on urlfetch.

How can I force urlfetch to use TLS 1.1 or otherwise work around this bug? When I upload my app to GAE it works fine, so I only need a local fix for my SDK. I have tried replacing the custom httplib.py with a patched stock httplib.py from Python 2.7, but this just leads to further problems as that module needs ssl and socket, which in turn needs another os, etc.

EDIT: Last part of traceback:

  File "evernote-sdk/lib/thrift/transport/THttpClient.py", line 122, in flush
    self.code, self.message, self.headers = self.__http.getreply()
  File "google_appengine/google/appengine/dist27/httplib.py", line 592, in getreply
    response = self._conn.getresponse()
  File "google_appengine/google/appengine/dist27/httplib.py", line 502, in getresponse
    raise HTTPException(str(e))
HTTPException: ApplicationError: 6 [Errno 8] _ssl.c:504: EOF occurred in violation of protocol

It is the combination of Ubuntu 12.04, Evernote's server/SDK and Google App Engine SDK that is the problem.

Answer 1

The traceback shows the error originates here: http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/dist27/httplib.py#490 This code block uses self._fetch which is assigned to be urlfetch.fetch in line 363 of the same file.

Now that we now it uses urlfetch and since the problem is only on dev_appserver, let's have a look at urlfetch_stub.py. The stub module uses the fancy_urllib library to make requests. We also know that the ubuntu launchpad bug suggests to do a

< self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
---
> self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file, ssl_version=ssl.PROTOCOL_TLSv1)

which hints us at the call we want to patch. Looking for 'ssl.wrap_socket' in fancy_urllib gives us 1 hit: init.py#159">http://code.google.com/p/googleappengine/source/browse/trunk/python/lib/fancy_urllib/fancy_urllib/init.py#159

Can you try to patch this method? I can't verify as I don't know how to reproduce the issue.

FWIW, searching for "ssl.wrap_socket" over all the SDK code also points at a couple of lines in the httplib2 third-party library, but right now I don't think this is what needs patching.