Python 27 can't import shared object from guppy

Question

I installed guppy the memory profiler from its svn#95 via "sudo python setup.py install".

It looks properly installed.

yey@yey:/usr/local/lib/python2.7/dist-packages/guppy/heapy$ ls *.so *.py
AbstractAlgebra.py  ImpSet.py          Path.py         Remote.py  Use.py
Classifiers.py      __init__.py        pbhelp.py       RM.py      View.py
Console.py      Monitor.py         Prof.py         Spec.py
Doc.py          OutputHandling.py  RefPat.py       Target.py
heapyc.so       Part.py        RemoteConstants.py  UniSet.py

But I still can't import it. Guppy's Python source does this import so it should succeed.

>>> import guppy.heapy
>>> import guppy.heapy.heapyc
# trying /usr/local/lib/python2.7/dist-packages/guppy/heapy/heapyc.so
# trying /usr/local/lib/python2.7/dist-packages/guppy/heapy/heapycmodule.so
# trying /usr/local/lib/python2.7/dist-packages/guppy/heapy/heapyc.py
# trying /usr/local/lib/python2.7/dist-packages/guppy/heapy/heapyc.pyc
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named heapyc

My question is, Python clearly made an attempt to import the file at the correct location. Why did it fail? Is it because the .so file corrupted? Or is it my ld.so.cache bad somehow? Thanks!

Answer 1

There are many possible problems with the .so file that could cause this—no read access, a corrupted file, an empty file, a perfectly valid library but for the wrong platform/architecture, etc. Worse, the .so itself may be fine, but it may have load-time dependencies on a different file that has any of the above problems.

Unfortunately, the Python 2.x importer doesn't show you which problem it's actually hit; all you can tell is that, for some reason, the call to open the shared library failed.

It's worth noting that in 3.1 or later, you would have gotten a much more useful error message, something like this:

ImportError: dlopen(/usr/local/lib/python3.3/dist-packages/guppy/heapy/heapyc.so, 2): no suitable image found.  Did find:
    /usr/local/lib/python3.3/dist-packages/guppy/heapy/heapyc.so: Permission denied

However, that's only possible because the importer was rewritten from scratch for 3.1, and there's no way such a radical change is ever going to be backported to 2.7.

---

Most platforms come with tools that let you test shared libraries, and this is really the best way to diagnose the problem.

But for a simple and platform-independent test, you can just use the ctypes library that comes with Python itself:

>>> import ctypes
>>> ctypes.CDLL('/usr/local/lib/python2.7/dist-packages/guppy/heapy/heapyc.so')

You should get an error, like this:

OSError: /usr/local/lib/python2.7/dist-packages/guppy/heapy/heapyc.so: cannot open shared object file: Permission denied

In this case, the file isn't readable (or, on platforms that require shared libraries to be executable, it either isn't readable or isn't executable), which should be enough to fix the problem. So, a chmod a+r should fix it (although you may want to go further and figure out why it wasn't readable in the first place).

If the error doesn't tell you enough to fix it yourself, and searching doesn't help, at least you can come to SO and ask a question that will be much more likely to get an immediate answer…