Streaming with gunicorn

Question

I'm trying to stream data from a flask/gunicorn server:

while (True):
    result = json.dumps(tweetQueue.get())
    yield result

However, 30 seconds into the stream, gunicorn times out my connection and stops the stream. How can I make the timeout such that publishing new data to the stream from the server will restart the timeout so the stream will not be terminated?

Thanks!

Answer 1

I am answering my own question after doing some more research.

gunicorn server:app -k gevent

This uses asynchronous workers, which have the benefit of using Connection: keep-alive when serving requests. This allows the request to be served indefinitely.

Answer 2

Consider using the built-in BaseHTTPServer instead of gunicorn. The following example launches 100 handler threads on the same port, with each handler started through BaseHTTPServer. It streams fine, supports multiple connections on 1 port, and generally runs 2X faster than gunicorn too. And you can wrap your socket in SSL if you want that too.

import time, threading, socket, SocketServer, BaseHTTPServer

class Handler(BaseHTTPServer.BaseHTTPRequestHandler):

    def do_GET(self):
        if self.path != '/':
            self.send_error(404, "Object not found")
            return
        self.send_response(200)
        self.send_header('Content-type', 'text/html; charset=utf-8')
        self.end_headers()

        # serve up an infinite stream
        i = 0
        while True:
            self.wfile.write("%i " % i)
            time.sleep(0.1)
            i += 1

# Create ONE socket.
addr = ('', 8000)
sock = socket.socket (socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(addr)
sock.listen(5)

# Launch 100 listener threads.
class Thread(threading.Thread):
    def __init__(self, i):
        threading.Thread.__init__(self)
        self.i = i
        self.daemon = True
        self.start()
    def run(self):
        httpd = BaseHTTPServer.HTTPServer(addr, Handler, False)

        # Prevent the HTTP server from re-binding every handler.
        # https://stackoverflow.com/questions/46210672/
        httpd.socket = sock
        httpd.server_bind = self.server_close = lambda self: None

        httpd.serve_forever()
[Thread(i) for i in range(100)]
time.sleep(9e9)

If you insist on using gunicorn anyway, remember to put it (and all its related packages: wsgi, gevent, flask) in a virtualenv to avoid conflicts with other software.

Answer 3

Gunicorn processes are sending "messages" to master process to let it know they are still alive (see https://github.com/benoitc/gunicorn/blob/master/gunicorn/workers/workertmp.py#L40). However this is not done during response serving (for example see https://github.com/benoitc/gunicorn/blob/master/gunicorn/workers/sync.py#L160) so if it takes longer then timeout the master process kills the worker.