Django fails to find R when loading RPy2 under virtualenv

Question

This is kind of confusing, apologies in advance. I am configuring a new (temporary) internal production web server with the following specs:

OSX 10.8.4 (Mountain Lion), Apache 2.2.25, PostgreSQL 9.2, Python 2.7.5, Django 1.5.2, R 3.0.1, RPy2 2.3.7, virtualenv 1.10.4, virtualenvwrapper 4.1.1, mod_wsgi 3.4

This was set up on a new machine using Homebrew and pip. I removed original, Apple-installed versions of Apache and Postgres before doing the installs. Both Apache and mod_wsgi work. mod_wsgi is installed to run as daemon process.

Virtual hosts is configured in Apache to access django via mod_wsgi and the wsgi.py file contains the following lines:

import os,sys,site
site.addsitedir('/Users/kaiju/.virtualenvs/nauru_dev/lib/python2.7/site-packages')

# Activate the virtual environment
# Taken from http://thecodeship.com/deployment/deploy-django-apache-virtualenv-and-mod_wsgi/
activate_env = os.path.expanduser('/Users/kaiju/.virtualenvs/nauru_dev/bin/activate_this.py')
execfile(activate_env,dict(__file__=activate_env))

All python packages (including django and RPy2) have been installed using the virtual environment's python site-packages.

R was installed using Homebrew and works fine from the command line or from python interpreter when invoked within the virtual env (most of the time, more later).

homebrew.mxcl.httpd.plist has been added to /Library/LaunchDaemons instead of ~/Library/LaunchAgents so that it would automatically start when the machine is rebooted. homebrew.mxcl.postgres.plist has been added to ~/Library/LaunchAgents (probably should also move it to /Library/LaunchDaemons).

So here is the problem. When everything was installed, it worked together fine. To verify everything was cool, I restarted the machine and tested the django application. Apache is running, as is django, but now I get the following error:

RuntimeError at /admin/
R_HOME not defined, and no R command in the PATH.
     Request Method:    GET
        Request URL:    http://nauru.xoma.com/admin/
     Django Version:    1.5.2
     Exception Type:    RuntimeError
    Exception Value:    R_HOME not defined, and no R command in the PATH.
    Exception Location: /Users/kaiju/.virtualenvs/nauru_dev/lib/python2.7/site-packages/rpy2/rinterface/__init__.py in <module>, line 48
  Python Executable:    /usr/local/opt/python/bin/python2.7
     Python Version:    2.7.5
        Python Path:    ['usr/local/var/django/code/ri',
                         '/usr/local/var/django/code',
                         '/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools-0.9.8-py2.7.egg',
                         '/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.4-py2.7.egg',
                         '/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
                         '/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
                         '/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
                         '/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
                         '/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
                         '/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
                         '/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',
                         '/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload',
                         '/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages',
                         '/Library/Python/2.7/site-packages',
                         '/Users/kaiju/.virtualenvs/nauru_dev/lib/python2.7/site-packages',
                         '/Users/kaiju/.virtualenvs/nauru_dev/lib/python2.7/site-packages/PIL']

But if I open a terminal window and type

sudo apachectl stop
sudo apachectl start

When the server comes back, everything works. This does not happen if I use sudo apachectl restart, in that case the error remains. What is going on? Too many moving parts for me.

Answer 1

The best way to resolve this issue is to add

os.environ['R_HOME']='/usr/local/Cellar/r/3.0.1/R.framework/Resources'

to the beginning of the wsgi.py file that is being used for mod_wsgi / django. It should be added before activating the virtual environment. Any other environ variables should also be added here. So now the wsgi.py file looks like this:

import os,sys,site
os.environ["R_HOME"]='/usr/local/Cellar/r/3.0.1/R.framework/Resources'
site.addsitedir('/Users/kaiju/.virtualenvs/nauru_dev/lib/python2.7/site-packages')

# Activate the virtual environment
# Taken from http://thecodeship.com/deployment/deploy-django-apache-virtualenv-and-mod_wsgi/
activate_env = os.path.expanduser('/Users/kaiju/.virtualenvs/nauru_dev/bin/activate_this.py')
execfile(activate_env,dict(__file__=activate_env))

sys.path.insert(0,'/usr/local/var/django/code')
sys.path.insert(0,'/usr/local/var/django/code/ri')

# We defer to a DJANGO_SETTINGS_MODULE already in the environment. This breaks
# if running multiple sites in the same mod_wsgi process. To fix this, use
# mod_wsgi daemon mode with each site in its own daemon process, or use
# os.environ["DJANGO_SETTINGS_MODULE"] = "ri.settings"
os.environ.setdefault("DJANGO_SETTINGS_MODULE","ri.settings")

# This application object is used by any WSGI server configured to use this
# file. This includes Django's development server, if the WSGI_APPLICATION
# setting points here.
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

Many thanks to those who contributed to this discussion!

Answer 2

I believe that sys.path has little to do with the environment variable PATH, which the one that should be set (along with LD_LIBRARY_PATH if needed).

I am guessing that Apache is running as a dedicated user (not your own UID for obvious security reasons), and that user does not have R in its path (other security step - by default Apache should not be able to run any executable on your system).

You'll have to either allow Apache to run the system's R, or have an R install only for your server (the later being the better if considering anything beyond a prototype to play with)