uWSGI + Django + Virtualenv unable to pick up _functools (import error)

Question

OK, So I have tried this with & without a virtualenv:

uwsgi --home /home/auston/new_proj/ --socket /tmp/uwsgi2.sock --chmod-socket --module app_wsgi --pp /home/auston/new_proj/nikeshere --logto /tmp/uwsgi.log --master --processes 4 -P

Pretty much no matter what, I get this:

*** Starting uWSGI 0.9.6.5 (32bit) on [Thu Oct 21 08:05:44 2010] ***
compiled with version: 4.4.3
Python version: 2.6.6 (r266:84292, Oct 21 2010, 04:07:38)
[GCC 4.4.3]
your memory page size is 4096 bytes
allocated 412 bytes (0 KB) for 1 request's buffer.
Setting PythonHome to /home/auston/new_proj/...
binding on UNIX socket: /tmp/uwsgi2.sock
chmod() socket to 666 for lazy and brave users
your server socket listen backlog is limited to 64 connections
added /home/auston/new_proj/nikeshere to pythonpath.
initializing hooks...done.
['/home/auston/new_proj/nikeshere', '.', '', '/home/auston/new_proj/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg', '/home/auston/new_proj/lib/python2.6/site-packages/pip-0.8.1-py2.6.egg', '/home/auston/new_proj/lib/python26.zip', '/home/auston/new_proj/lib/python2.6', '/home/auston/new_proj/lib/python2.6/plat-linux2', '/home/auston/new_proj/lib/python2.6/lib-tk', '/home/auston/new_proj/lib/python2.6/lib-old', '/home/auston/new_proj/lib/python2.6/lib-dynload', '/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2', '/usr/lib/python2.6/lib-tk', '/home/auston/new_proj/lib/python2.6/site-packages', '/usr/local/lib/python2.6/dist-packages/pip-0.8.1-py2.6.egg', '/usr/local/lib/python2.6/site-packages', '/usr/local/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages', '/home/auston/new_proj/nikeshere', '/usr/local/lib/python2.6']
Traceback (most recent call last):
  File "/home/auston/new_proj/nikeshere/app_wsgi.py", line 11, in <module>
    import django.core.handlers.wsgi
  File "/usr/local/lib/python2.6/site-packages/django/core/handlers/wsgi.py", line 1, in <module>
    from threading import Lock
  File "/usr/lib/python2.6/threading.py", line 13, in <module>
    from functools import wraps
  File "/usr/lib/python2.6/functools.py", line 10, in <module>
    from _functools import partial, reduce
ImportError: No module named _functools

If I change --home to /usr/local/lib/python/2.6 I get fail on my app_wsgi.py import of os. Here it is, below, just in case:

import sys
import os

sys.path.append(os.path.abspath(os.path.dirname(__file__)))

import django.core.handlers.wsgi

application = django.core.handlers.wsgi.WSGIHandler()

Essentially I am asking, how can I get uWSGI to recognize functools OR get on the right path (path is in output above). I would appreciate any help you guys can give!!

P.S. Ubuntu 10.04 - uWSGI 0.9.6.5 - NGINX 0.8.53 - virtual env Python 2.6.5 - "regular (or system)" Python 2.6.6 - Django 1.2.3

UPDATE:

I was able to get uwsgi to start accepting requests if I omit the "--module" like so:

uwsgi --home /home/auston/new_proj --socket /tmp/uwsgi2.sock --chmod-socket --pp /home/auston/new_proj/nikeshere --logto /tmp/uwsgi.log --master --processes 4 -P

but now I get a app not found error:

"uWSGI Error wsgi application not found"

I'm closer but I would still appreciate suggestions as the app is not found because i cannot include the module needed to load it!

Answer 1

So as noted above, the problem has been with the pythonpath & it's inability to find a module named _functools.

Apparently, _functools is a c module & I needed to append the it's path to the pythonpath in order for it to be found, so the difference from the original wsgi.py, is now:

import sys
sys.path.append('/usr/local/lib/python2.6/lib-dynload') # to load _functools
sys.path.append('/usr/local/lib/python2.6/site-packages') # to load django
sys.path.append('/usr/local/lib/python2.6/dist-packages') # cautionary to load django
sys.path.append('/usr/lib/python2.6') # to load os
import os

os.environ['DJANGO_SETTINGS_MODULE'] = 'iwin.settings'

import django.core.handlers.wsgi

application = django.core.handlers.wsgi.WSGIHandler()

Very hacky, but it works for now...

Answer 2

I know it's old topic and versions of stack building blocks changed, but I had the same problem with not recognizing under uWSGi installed libs in virtualenv. The solution is to point home parameter to virtualenv, as shown below (taken from https://uwsgi.readthedocs.org/en/latest/tutorials/Django_and_nginx.html).

So for me command:

uwsgi --http :8000 --module ii.wsgi --home /home/dev/.virtualenvs/ii_env/

worked, while being in the django application (ii) directory.

# mysite_uwsgi.ini file
[uwsgi]

# Django-related settings
# the base directory (full path)
chdir           = /path/to/your/project
# Django's wsgi file
module          = project.wsgi
# the virtualenv (full path)
home            = /path/to/virtualenv

# process-related settings
# master
master          = true
# maximum number of worker processes
processes       = 10
# the socket (use the full path to be safe
socket          = /path/to/your/project/mysite.sock
# ... with appropriate permissions - may be needed
# chmod-socket    = 664
# clear environment on exit
vacuum          = true

Answer 3

Check out http://blog.zacharyvoase.com/2010/03/05/django-uwsgi-nginx/. He's using very similar set up.

Answer 4

I've been having very similar problem and I found this:

When you install virtuelenv, it 'installs' the Python standard library by creating symlinks to the original one (in like /usr/lib/python2.7). But when you check your virtualenv Python lib directory, there are symlinks created for only a few basic libraries. Your functools is probably not among them.

So the solution is to create the symlink manually. It is a PITA, because you may have to create a lot of symlinks, but it seems like a cleaner solution to me. You don't have to hack any source files and it's transparent.

The symlink should be created not in the root of the venv_directory, but in e.g.

venv_directory/lib/python2.7/site-packages/

Hope it works for you!