trying to import a *.pyc as a module

Question

I have a python script that is trying to import another script somewhere in the file-system (path is only known at runtime). To my understanding I need to use the imp module and this might work, but when loading the module I get an error that modules used by the imported module are not found.

Heres the code:

importer.py:

import imp
imp.load_compiled("my_module","full_path_to_my_module\\my_module.pyc")

my_module.py:

import sys
import another_module

When I run importer.py I get htis error message:

ImportError: No module named another_module

Whats going wrong here ? I suspect that when 'importer.py' is loading 'my_module.pyc' hes also trying to load 'another_module' (thats good) but is looking in the wrong place (eg not 'full_path_to_my_module')

EDIT: I tried adding 'full_path_to_my_module' to the system path:

import imp
import sys

sys.path.append(full_path_to_my_module)
imp.load_compiled("my_module",full_path_to_my_module+my_module)

But I still get the same error

Maybe I do something thats not necessary - Here's my goal: I want to be able to use all functionality of 'my_module.pyc' inside 'importer.py'. But the location of 'my_module.pyc' is given as a parameter to 'importer.py'.

Answer 1

imp.load_compiled returns the compiled module object, it is different to the import statement which also binds the module to a name

import imp
my_module = imp.load_compiled("my_module", "full_path_to_my_module/my_module.pyc")

Then you can do something like:

my_module.yayfunctions('a')

Complete example session:

$ cat /tmp/my_module.py
def yayfunctions(a):
    print a
$ python -m compileall /tmp/my_module.py
$ ls /tmp/my_module.py*
my_module.py   my_module.pyc
$ python
>>> import imp
>>> my_module = imp.load_compiled("my_module", "/tmp/my_module.pyc")
>>> my_module.yayfunctions('a')
a

---

Edit regarding comment (ImportError: No module named another_module), I assume the error is caused by the code in my_module.pyc, and the another_module.py lives in the same directory

In that case, as others have suggested, it's simpler to just add the directory containing my_module to sys.path and use the regular import mechanism, specifically __import__

Here's a function which should do what you want:

import os


def load_path(filepath):
    """Given a path like /path/to/my_module.pyc (or .py) imports the
    module and returns it
    """

    path, fname = os.path.split(filepath)
    modulename, _ = os.path.splitext(fname)

    if path not in sys.path:    
        sys.path.insert(0, path)

    return __import__(modulename)


if __name__ == '__main__':
    # Example usage
    my_module = load_path('/tmp/my_module.py')
    my_module.yayfunctions('test')

Answer 2

It is since at the scope of import another_module your "full_path_to_my_module" isn't known.

Have you tried to add the path to known paths instead, i.e.:

import sys
sys.path.append("full_path_to_my_module")

Answer 3

You don't actually need to use the imp module to load pyc modules.

An easy way to try it out is to make two python modules, one importing from the other and run it. Delete then the imported .py file so you only get the .pyc file left: when running the script the import will work just fine.

But, for importing py files from random directories, you may want to add that directory to the python path first before importing it.

For instance:

import sys
sys.path.insert(0, "/home/user/myrandomdirectory")

Answer 4

Loading pyc files works the exact same way as loading a py file except it doesn't do a compile step. Thus just using import mymodule will work as long as the version number of the pyc is the same as the python you're running. Otherwise you'll get a magic number error.

If you module isn't in your path you'll need to add that to sys -- or if its a subdirectory, add a __init__.py file to that directory..