LD_LIBRARY_PATH precendence and trouble with compiling

Question

I am trying to install some software on a Linux machine (python's rpy2 package, rpy2.robjects in particular, if it matters). I need the software to look for its shared libraries in my local space, not in the global spaces like /usr/lib64 whatever. I do not have admin privileges on this machine. My problem is that even though I set LD_LIBRARY_PATH to point to my local space, the software still goes to the /usr/lib64/whatever, where it finds libraries that are out of date. These libraries are missing some objects it needs, so it fails. What could be taking precedence over LD_LIBRARY_PATH, and is there a way to change/get rid of it? Thanks.

BTW, someone on this site had a similar question a year or more ago, but his answer involved the env variable RPATH, and changing it with the chrpath utility. It is not clear to me at all that rpy2 is using RPATH, and chrpath seems unavailable on my system.

Addendum: I tried running with LD_DEBUG=libs. Got alot of output that looks like the system is looking for the libs in my LD_LIBRARY_PATH and finding them. Here is where the trouble seems to start:

/usr/local/lib64/R/library/methods/libs/methods.so: error: symbol lookup error:
undefined symbol: Rf_allocS4Object (fatal)
Error in dyn.load(file, DLLpath = DLLpath, ...) : 
unable to load shared object '/usr/local/lib64/R/library/methods/libs/methods.so':
/usr/local/lib64/R/library/methods/libs/methods.so: undefined symbol: Rf_allocS4Object

So my guess is that the trouble is that whatever is in DLLpath is overriding LD_LIBRARY_PATH. I have tried to change this by prepending my directories to os.environ['PATH'], but no do. There is apparently no "DLLPATH", as I thought there would be.

OK, that's progress, I guess. Anybody have anything else? Thanks.

Answer 1

If anyone is still reading this, I engaged in some "personnel engineering" to solve the problem, i.e. got the system admins to re-install R so that it had everything I needed. Was certainly nice of them. Thanks very much to everyone who gave suggestions. Would like to keep going on some of them, but I've got to get busy on this project. Much obliged!

Answer 2

Have a look at a file named $R_HOME/etc/ldpaths (where in your case $R_HOME seems to be /usr/local/lib64/R). It is the commands in this file that set LD_LIBRARY_PATH at R's start-up.

Mine looks like this:

flodel@netbook-samsung-N150:~$ cat /usr/lib/R/etc/ldpaths 
: ${JAVA_HOME=/usr/lib/jvm/java-6-openjdk/jre}
: ${R_JAVA_LD_LIBRARY_PATH=${JAVA_HOME}/lib/i386/client:${JAVA_HOME}/lib/i386:/usr/lib/jni}
if test -n ""; then
: ${R_LD_LIBRARY_PATH=${R_HOME}/lib:}
else
: ${R_LD_LIBRARY_PATH=${R_HOME}/lib}
fi
if test -n "${R_JAVA_LD_LIBRARY_PATH}"; then
  R_LD_LIBRARY_PATH="${R_LD_LIBRARY_PATH}:${R_JAVA_LD_LIBRARY_PATH}"
fi
if test -z "${LD_LIBRARY_PATH}"; then
  LD_LIBRARY_PATH="${R_LD_LIBRARY_PATH}"
else
  LD_LIBRARY_PATH="${R_LD_LIBRARY_PATH}:${LD_LIBRARY_PATH}"
fi
export LD_LIBRARY_PATH

If you do not have write access to the file, you can still do this before starting R:

export R_LD_LIBRARY_PATH=/your/custom/path

I tested on my machine that it works by running the following after R is started:

Sys.getenv("LD_LIBRARY_PATH")
#[1] "/your/custom/path:/usr/lib/jvm/java-6-openjdk/jre/lib/i386/client:/usr/lib/jvm/java-6-openjdk/jre/lib/i386:/usr/lib/jni"

Answer 3

RPATH is only useful when compiling (well, linking); it affects the library search path that gets baked into the binary.

Try running with LD_DEBUG=libs, which will show libraries are being loaded from which paths. Is it trying to load from your LD_LIBRARY_PATH but failing, or not searching there in the first place (perhaps rpy2 has its own library path mechanism?), or something else?

Answer 4

Try adding your directory on the left of the LD_LIBRARY_PATH as the precedence goes left to right.

export LD_LIBRARY_PATH=~/your/custom/path:$LD_LIBRARY_PATH