Unable to import installed packages in python after updating OSX and python

Question

I recently updated my system (OXS 10.9.1 -> 10.9.2), and my python using brew (2.7.3 -> 2.7.6).

Now I am getting very strange behaviour. One example is trying to run a test app which creates flask sockets:

jono@air:~/Workspace/sandbox $ python app.py
Traceback (most recent call last):
  File "app.py", line 9, in <module>
    import flask, flask_sockets, time, threading, simplejson
  File "/usr/local/lib/python2.7/site-packages/flask/__init__.py", line 21, in <module>
    from .app import Flask, Request, Response
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 26, in <module>
    from . import json
  File "/usr/local/lib/python2.7/site-packages/flask/json.py", line 25, in <module>
    from itsdangerous import json as _json
ImportError: No module named itsdangerous

First thought is to update itsdangerous:

jono@air:~/Workspace/sandbox $ pip install itsdangerous --upgrade
Requirement already up-to-date: itsdangerous in /usr/local/lib/python2.7/site-packages
Cleaning up...

Checking that it installed correctly:

jono@air:~/Workspace/sandbox $ locate itsdangerous
/usr/local/lib/python2.7/site-packages/itsdangerous-0.23-py2.7.egg-info
/usr/local/lib/python2.7/site-packages/itsdangerous-0.23-py2.7.egg-info/PKG-INFO
/usr/local/lib/python2.7/site-packages/itsdangerous-0.23-py2.7.egg-info/SOURCES.txt
/usr/local/lib/python2.7/site-packages/itsdangerous-0.23-py2.7.egg-info/dependency_links.txt
/usr/local/lib/python2.7/site-packages/itsdangerous-0.23-py2.7.egg-info/installed-files.txt
/usr/local/lib/python2.7/site-packages/itsdangerous-0.23-py2.7.egg-info/not-zip-safe
/usr/local/lib/python2.7/site-packages/itsdangerous-0.23-py2.7.egg-info/top_level.txt
/usr/local/lib/python2.7/site-packages/itsdangerous.py
/usr/local/lib/python2.7/site-packages/itsdangerous.pyc

Checking my PYTHONPATH:

jono@air:~/Workspace/sandbox $ echo $PYTHONPATH
/usr/local/lib/python2.7/site-packages/

Checking that python is looking in the correct PYTHONPATH:

jono@air:~/Workspace/sandbox $ python
Python 2.7.6 (default, Mar  6 2014, 10:46:26) 
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import itsdangerous
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named itsdangerous
>>> import os
>>> os.getenv("PYTHONPATH")
'/usr/local/lib/python2.7/site-packages/'
>>> 

I'm having similar issues with other basic packages like virtualenv too:

jono@air:~/Workspace/sandbox $ virtualenv venv
Traceback (most recent call last):
  File "/usr/local/bin/virtualenv", line 5, in <module>
    from pkg_resources import load_entry_point
  File "build/bdist.macosx-10.9-x86_64/egg/pkg_resources.py", line 2720, in <module>

  File "build/bdist.macosx-10.9-x86_64/egg/pkg_resources.py", line 588, in resolve
    The `plugin_env` should be an ``Environment`` instance that contains
pkg_resources.DistributionNotFound: virtualenv==1.9.1

Is it installed?

jono@air:~/Workspace/sandbox $ pip install virtualenv --upgrade
Requirement already up-to-date: virtualenv in /usr/local/lib/python2.7/site-packages
Cleaning up...

But strangely this time python can see it:

jono@air:~/Workspace/sandbox $ python
Python 2.7.6 (default, Mar  6 2014, 10:46:26) 
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import virtualenv
>>> dir(virtualenv)
['ACTIVATE_BAT', 'ACTIVATE_CSH', 'ACTIVATE_FISH', 'ACTIVATE_PS', 'ACTIVATE_SH', 'ACTIVATE_THIS', 'BIG_ENDIAN', 'ConfigOptionParser', 'ConfigParser', 'DEACTIVATE_BAT', 'DISTUTILS_CFG', 'DISTUTILS_INIT', 'FAT_MAGIC', 'LC_LOAD_DYLIB', 'LITTLE_ENDIAN', 'Logger', 'MH_CIGAM', 'MH_CIGAM_64', 'MH_MAGIC', 'MH_MAGIC_64', 'OK_ABS_SCRIPTS', 'REQUIRED_FILES', 'REQUIRED_MODULES', 'SITE_PY', 'UpdatingDefaultsHelpFormatter', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__version__', '_find_file', 'abiflags', 'base64', 'call_subprocess', 'change_prefix', 'codecs', 'convert', 'copy_required_modules', 'copyfile', 'copyfileordir', 'create_bootstrap_script', 'create_environment', 'default_config_file', 'default_storage_dir', 'distutils', 'errno', 'expected_exe', 'file_search_dirs', 'fileview', 'filter_install_output', 'find_wheels', 'fix_lib64', 'fix_local_scheme', 'fixup_egg_link', 'fixup_pth_and_egg_link', 'fixup_pth_file', 'fixup_scripts', 'get_installed_pythons', 'glob', 'install_activate', 'install_distutils', 'install_python', 'install_wheel', 'is_cygwin', 'is_darwin', 'is_executable', 'is_executable_file', 'is_jython', 'is_pypy', 'is_win', 'join', 'logger', 'logging', 'mach_o_change', 'main', 'majver', 'make_environment_relocatable', 'make_exe', 'make_relative_path', 'maxint', 'minver', 'mkdir', 'optparse', 'os', 'path_locations', 'py_version', 're', 'read_data', 'relative_script', 'resolve_interpreter', 'rmtree', 'shutil', 'strtobool', 'struct', 'subprocess', 'subst_path', 'sys', 'tarfile', 'tempfile', 'user_dir', 'virtualenv_version', 'writefile', 'zlib']
>>> virtualenv.is_darwin
True
>>> virtualenv.__file__
'/usr/local/lib/python2.7/site-packages/virtualenv.pyc'
>>> 

Answer 1

Somehow, many of my packages in /usr/local/lib/python2.7/site-packages/ had root ownership. This was causing a host of issues :/

sudo pip uninstall XXX

followed by

pip install XXX

Resolved most of the issues.

Let the lesson be learnt that one should never 'sudo pip install' anything.