Silencing broken pipe errors (`[Errno 32] Broken pipe`) in python with WSGI+Pyramid

Question

I have a rather simple, naive Python/WSGI/Pyramid web-server.

It's run using wsgiref.simple_server.make_server(), on a server built using pyramid.config.Configurator().make_wsgi_app(). This server works fine.

However, the application it's serving has a lot of javascript image mouseover popups. If you run the mouse across the page, it can generate 20+ image requests. This is fine as well (It's an internal thing, not a lot of users).

However, doing so causes the server to emit something like half a dozen error tracebacks:

10.1.1.4 - - [25/Apr/2014 01:56:42] "GET /*SNIP* 500 59
----------------------------------------
Exception happened during processing of request from ('10.1.1.4', 18338)
Traceback (most recent call last):
  File "/usr/lib/python3.4/wsgiref/handlers.py", line 138, in run
    self.finish_response()
  File "/usr/lib/python3.4/wsgiref/handlers.py", line 180, in finish_response
    self.write(data)
  File "/usr/lib/python3.4/wsgiref/handlers.py", line 274, in write
    self.send_headers()
  File "/usr/lib/python3.4/wsgiref/handlers.py", line 333, in send_headers
    self._write(bytes(self.headers))
  File "/usr/lib/python3.4/wsgiref/handlers.py", line 453, in _write
    self.stdout.write(data)
  File "/usr/lib/python3.4/socket.py", line 391, in write
    return self._sock.send(b)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.4/wsgiref/handlers.py", line 141, in run
    self.handle_error()
  File "/usr/lib/python3.4/wsgiref/handlers.py", line 368, in handle_error
    self.finish_response()
  File "/usr/lib/python3.4/wsgiref/handlers.py", line 180, in finish_response
    self.write(data)
  File "/usr/lib/python3.4/wsgiref/handlers.py", line 274, in write
    self.send_headers()
  File "/usr/lib/python3.4/wsgiref/handlers.py", line 331, in send_headers
    if not self.origin_server or self.client_is_modern():
  File "/usr/lib/python3.4/wsgiref/handlers.py", line 344, in client_is_modern
    return self.environ['SERVER_PROTOCOL'].upper() != 'HTTP/0.9'
TypeError: 'NoneType' object is not subscriptable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.4/socketserver.py", line 306, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib/python3.4/socketserver.py", line 332, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python3.4/socketserver.py", line 345, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python3.4/socketserver.py", line 666, in __init__
    self.handle()
  File "/usr/lib/python3.4/wsgiref/simple_server.py", line 126, in handle
    handler.run(self.server.get_app())
  File "/usr/lib/python3.4/wsgiref/handlers.py", line 144, in run
    self.close()
  File "/usr/lib/python3.4/wsgiref/simple_server.py", line 35, in close
    self.status.split(' ',1)[0], self.bytes_sent
AttributeError: 'NoneType' object has no attribute 'split'

I understand why I'm getting broken pipe errors (the request for the image is canceled before the image has fully transfered, because the mouseover popup has closed), and it seems harmless.

However, I have no idea how to silence this traceback. There are thousands of them in my logs, and it makes debugging actual errors a nightmare. I don't care that I'm getting broken pipe errors, how can I catch them and swallow them silently?

It seems like wsgiref.simple_server.make_server() installs an internal handler that catches BrokenPipeError: [Errno 32] Broken pipe, prints the traceback, and then swallows the error. I've tried wrapping the run_server() call in a try-except clause, and it doesn't have any effect.

Answer 1

I wound up just switching to using the CherryPy WSGI Server. It doesn't suffer from the broken pipe log issues, and is probably far more robust as well.

It also uses a threadpool, so it's more performant as well (multiple requests aren't blocking!).

Answer 2

I didn't find a straightforward way for achieving this, however, you can always do some monkey patching:

from wsgiref.handlers import BaseHandler
import sys


def ignore_broken_pipes(self): 
    if sys.exc_info()[0] != BrokenPipeError: BaseHandler.__handle_error_original_(self)

BaseHandler.__handle_error_original_ = BaseHandler.handle_error
BaseHandler.handle_error = ignore_broken_pipes

You will not see these annoyances after running this code somewhere in the beginning anymore.

For me, it looks like a bug somewhere in wsgiref implementation of BaseHandler:

def handle_error(self):
    """Log current error, and send error output to client if possible"""
    self.log_exception(sys.exc_info())
    if not self.headers_sent:
        self.result = self.error_output(self.environ, self.start_response)
        self.finish_response()
    # XXX else: attempt advanced recovery techniques for HTML or text?

if BrokenPipeError is handled by this method, finish_response crashes. Why do we ever want to finish response if pipe is broken? Where the data is sent to?