How to force importlib find_loader to use exclusively a specific path?

Question

I'm trying to re-implement some code based on the imp module to to use importlib instead. There is a subtle difference between the functions imp.find_module and importlib.find_loader:

# Using imp find_module
>>> import imp
>>> imp.find_module("abc", ["some/path/module.py"])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.3/imp.py", line 220, in find_module
    raise ImportError(_bootstrap._ERR_MSG.format(name), name=name)
ImportError: No module named 'abc'

# Using import find_loader
>>> import importlib
>>> loader = importlib.find_loader("abc", ["some/path/module.py"])
>>> loader.path
'/usr/lib64/python3.3/abc.py'

It looks like importlib is falling back to system libraries, whilst imp.find_module is using the path parameter in an exclusive way, which ends up raising an exception.

How to force importlib.find_loader to use exclusively a specific path?

Answer 1

I found that the answer is not to use find_loader but to use instead the importlib.machinery.PathFinder class to lookup for modules in specific paths.

Answer 2

It depends on what your exact goal is for your code. First off, both calls are wrong as the 'path' argument is meant to be a list of directories, not a specific file.

Second, abc is being returned from the standard library because it is already in sys.modules. Remember, importlib.find_loader() finds the loader for the specified module, so that includes looking in sys.modules['abc'].__loader__ to skip trying to find it needlessly. Had you specified a module in the standard library (e.g. tokenize) not already imported then importlib.find_loader() would have returned None.

In other words importlib.find_loader() will search for a module in a specific directory when specified in 'path', but you have to be looking for something that has not already been imported.

Answer 3

I think your usage of both imp and importlib may be incorrect. The list of paths should be paths to directories, not to specific .py files. You're never going to find abc.py inside module.py, so the import fails.

e.g. I had some custom Fabric pip functions defined in a local pip.py file, but I also want to load the official pip.py in my site-packages directory. Since I first use importlib to import my custom pip.py, this makes import pip load my custom version. To load the module in site-packages, I have to do:

from distutils.sysconfig import get_python_lib
import imp
real_pip = imp.load_module('pip', *imp.find_module('pip', [get_python_lib()]))