Web app hangs for several hours in ssl.py at self._sslobj.do_handshake()

Question

I am using Python 2.7.5. I have a web app which queries an API every few minutes and has been working successfully for the last day or so. However, after leaving it sitting for a few hours, I came back to find my program stalled with no activity for several hours. I quit the program and found that it has been stalled in the ssl handshake method for most of the day, during one of the API calls.

Here is the traceback:

...
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 143, in __init__
  self.do_handshake()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 305, in do_handshake
  self._sslobj.do_handshake()

I did a little research, and it seems that this was a problem with the SSL libraries in Python 2.6, but it has since been fixed. I'm wondering why my program got stuck here without throwing an exception or anything.

If there is a way to set a timeout for the SSL handshake method, I would gladly do that, as I would like to avoid my program being stopped indefinitely from something like this again. I am using the Requests HTTP library and this is running on Mac OSX 10.9, if that matters. Suggestions?

EDIT: I did some research, and it seems that others have had this specific problem with SSL, despite the "fix" implemented in 2.6. Not sure what the solution is yet, however. Any help is appreciated.

EDIT 3: Added my solution as an answer to this question.

Answer 1

After browsing through the Python section of Stack Overflow, I found something which may not fix the core issue causing the problem, but is definitely good enough to handle any situation in which this issue pops up. The following question has various solutions which will throw some sort of exception if a function takes too long to complete. That is how I solved this issue in the end. The top answer is UNIX-only, however there are some others which use threading and work on every platform:

Timeout function if it takes too long to finish

This is a strange issue which is honestly pretty hard to reproduce. I've only seen it twice after thousands and thousands of API calls. I think it's pretty unlikely that someone will arrive at a better solution than this one, which is sort of a hack, but definitely solves the problem. You can throw an exception, then either attempt the SSL connection again, or continue with another part of your program.

I think my answer will suffice for now, but if someone has something better, feel free to offer it forward. Honestly, it seems like the only fix for the underlying issue might be a bug fix in the actual ssl.py library, but it's impossible for me to say for sure.

Answer 2

I ran into this and was able to work around it using socket.setdefaulttimeout. socket.setdefaulttimeout modifies the behavior of all sockets created after calling it, which is convenient if the socket is encapsulated in library code. If you have access to the specific SSL socket you can probably use socket.settimeout.

Answer 3

In my case, the connection stuck in do_handshake(), I add a sleep before handshake, it seems solved.

def patch_ssl():
    import ssl
    import time
    def new_wrap_socket(self, sock, server_side=False,
                        do_handshake_on_connect=True,
                        suppress_ragged_eofs=True,
                        server_hostname=None, session=None):
        # SSLSocket class handles server_hostname encoding before it calls
        # ctx._wrap_socket()
        time.sleep(0.005)
        return self.sslsocket_class._create(
            sock=sock,
            server_side=server_side,
            do_handshake_on_connect=do_handshake_on_connect,
            suppress_ragged_eofs=suppress_ragged_eofs,
            server_hostname=server_hostname,
            context=self,
            session=session
        )
    ssl.SSLContext.old_wrap_sock = ssl.SSLContext.wrap_socket
    ssl.SSLContext.wrap_socket = new_wrap_socket
patch_ssl()