pip executes the wrong python library versions inside virtual env

Question

I create a virtual enviroment with virtualenvwrapper and then I try to install django in it with pip. However I keep getting an error due to a conflict in python versions.

$ mkvirtualenv env
$ workon env
$ pip install django
Downloading/unpacking django
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Users/mingot/virtualenvs/env/lib/python2.7/site-packages/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/Users/mingot/virtualenvs/env/lib/python2.7/site-packages/pip/commands/install.py", line 236, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/Users/mingot/virtualenvs/env/lib/python2.7/site-packages/pip/req.py", line 1085, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "/Users/mingot/virtualenvs/env/lib/python2.7/site-packages/pip/index.py", line 201, in find_requirement
    page = self._get_page(main_index_url, req)
  File "/Users/mingot/virtualenvs/env/lib/python2.7/site-packages/pip/index.py", line 554, in _get_page
    return HTMLPage.get_page(link, req, cache=self.cache)
  File "/Users/mingot/virtualenvs/env/lib/python2.7/site-packages/pip/index.py", line 671, in get_page
    resp = urlopen(url)
  File "/Users/mingot/virtualenvs/env/lib/python2.7/site-packages/pip/download.py", line 176, in __call__
    response = self.get_opener(scheme=scheme).open(url)
  File "/Users/mingot/virtualenvs/env/lib/python2.7/site-packages/pip/download.py", line 238, in get_opener
    headers.append(("User-agent", build_user_agent()))
  File "/Users/mingot/virtualenvs/env/lib/python2.7/site-packages/pip/download.py", line 35, in build_user_agent
    _implementation = platform.python_implementation()
  File "/Users/mingot/soft/anaconda/lib/python2.7/platform.py", line 1486, in python_implementation
    return _sys_version()[0]
  File "/Users/mingot/soft/anaconda/lib/python2.7/platform.py", line 1451, in _sys_version
    repr(sys_version))
ValueError: failed to parse CPython sys.version: '2.7.5 (default, Aug 25 2013, 00:04:04) \n[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)]'

In the system, I am running python anaconda:

$ python 
Python 2.7.5 |Anaconda 1.8.0 (x86_64)| (default, Oct 24 2013, 07:02:20) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin

and the $PATH is set to

/Users/mingot/soft/anaconda/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin

while inside the virtual enviroment, the python version is:

(env)$ python
Python 2.7.5 (default, Aug 25 2013, 00:04:04) 
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin

and the $PATH:

/Users/mingot/virtualenvs/env/bin:/Users/mingot/soft/anaconda/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin

I understand that the problem is that inside the virtual enviroment, while executing the non anaconda python 2.7.5, it is still using platforms.py from anaconda library, causing the crash in the evaluation of the regex as suggested here. I do not care which python version to use inside the virtual enviroment. Any suggestion on how to tell the python of inside the virtual enviroment the correct platforms.py to use?

Thanks!

Answer 1

I ran into the same issue on a Mac with Anaconda installed. The way I fixed it was searching for platform.py, then modifying the following (commenting out a line):

ORIGINAL

_sys_version_parser = re.compile(
    r'([\w.+]+)\s*'
    '\|[^|]*\|\s*' # version extra
    '\(#?([^,]+),\s*([\w ]+),\s*([\w :]+)\)\s*'
    '\[([^\]]+)\]?')

CHANGE TO

_sys_version_parser = re.compile(
    r'([\w.+]+)\s*'
    #'\|[^|]*\|\s*' # version extra # Just comment this line out
    '\(#?([^,]+),\s*([\w ]+),\s*([\w :]+)\)\s*'
    '\[([^\]]+)\]?')

This is with virtualenv==1.10.1 and anaconda 1.9.2

Answer 2

The solution I end up doing is creating a symbolic link to platforms.py inside the virtual enviroment library folder from the library file of the correct python version.

$ pwd
/Users/mingot/virtualenvs/env/lib/python2.7
$ ln -s /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/platform.py platform.py

Answer 3

The solution I worked out was to supplement virtualenv.py so it not only copies over the Python binary to the new environment, but also the accompanying libpythonX.Y.so shared library files. Without them, the tiny Anaconda Python binary (take a look at it — it is only a few kilobytes in size) links against the system libpython instead of the Anaconda libpython and so you have the system Python’s version string getting parsed by Anaconda's mutant platform.py code.

Here is a short shell script that will fix virtualenv.py; run it once you have your Anaconda environment activated and have run pip install virtualenv to make the virtualenv tool available inside of it:

https://github.com/brandon-rhodes/homedir/blob/master/bin/%2Cfix-virtualenv-for-anaconda