django with nginx + uwsgi

Question

I am trying django on nginx + uwsgi. It works very well (faster than apache mod_wsgi), but if I have more than 100 concurrent connexion ( ie : tested with ab -n 100000 -c 150 http://localhost:8081/ ), I have some broken pipe on uwsgi logs :

nginx.conf :

user  myuser;
worker_processes  8;

events {
    worker_connections  30000;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    keepalive_timeout  65;
    upstream django {
      ip_hash;
      server unix:/home/myuser/tmp/uwsgi.sock;

    }

    server {
        listen       8081;
        server_name  localhost;
        location / {
            uwsgi_pass  django;
            include     uwsgi_params;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

uwsgi is started like that :

/usr/local/bin/uwsgi -s /home/myuser/tmp/uwsgi.sock --pp /home/myuser/projects/django/est/nginx --module django_wsgi -L -l 500 -p 8

And the error messages from uwsgi are :

writev(): Broken pipe [plugins/python/wsgi_headers.c line 206]
write(): Broken pipe [plugins/python/wsgi_subhandler.c line 235]

version are : 1.0.6 for nginx and 0.9.9.2 for uwsgi

Do you know how to solve these error messages ?

Answer 1

I found the solution, The problem is not at uwsgi side, there is a linux limitation : socket are 128 request long, so to enlarge the waiting queue, you have to tune the kernel :

ie :

echo 3000 > /proc/sys/net/core/netdev_max_backlog
echo 3000 > /proc/sys/net/core/somaxconn

Answer 2

150 connection for 8 workers with a listen queue of 100 could be a too high value. Probably you only need to increase the listen queue. This is showed in the uWSGI homepage (under the benchmark sections)