cascaded library dependencies

Question

For reasons too lengthy to explain here, I need to create, in "C", a shared library (call it libA) that is callable within Python. This shared library needs to call another custom shared library (we'll name it libCuda). In addition libA calls many other external shared libraries such as MySQL, etc.

Before the inclusion of libCuda, all C-functions called within Python worked flawlessly, MySQL sub-calls worked fine, etc. However, once libCuda was added to libA, Python complains:

caughtError c process - [directory]/libA.so: undefined symbol: cudaFunction1

libCuda, as you might have guessed, is nVidia Cuda Code compiled with nvcc and setup to be C-linkable (i.e. export "C" in function prototyes) I have tested this shared library (libCuda) with a stand-alone test bench (written in C) and everything works as expected. This is the instruction used to compile the Cuda library:

nvcc -arch=sm_30 -shared -o libCuda.so *.cu -Xcompiler -fPIC

if I use:

$ ldd libA

I do not see any information indicating that libA needs to load libCuda (or MySQL for that matter)

The command used to compile libA is as follows (libCuda lives in local directory):

gcc *.c -c -L. -lCuda -lmysqlclient [many other shared libraries] -fPIC
gcc -shared -Wl,-soname,libA.so -lCuda -lmysqlclient [many other shared libraries] -o libA.so *.o

I have tried placing both library files in /usr/lib and explicitly exporting the LD_LIBRARY_PATH. No luck. Any help would be greatly appreciated!

Answer 1

Ok, so didn't win, but I was able to change the rules.

Using this as a reference, I first compiled the CUDA code with nvcc, thus creating several object ( *.o ) files. These object files were then added to the list of object files linked by gcc when creating the library, libA. In addition, the follwing linker arguments were added to the gcc command "-L/usr/local/cuda/lib64 -lcudart" (I'm using an x64 machine).

It is EXTREMELY IMPORTANT to point out that you must place any library dependency AFTER the object file that needs it. If you don't gcc will complain about undefined references. A good rule of thumb is to place all libraries at the end of your gcc line. See below for details.

In a nutshell here's what worked:

CUDA:

nvcc -arch=sm_30 -c *.cu -Xcompiler -fPIC

C:

gcc *.c -c -fPIC -L/usr/local/cuda/lib64 -lcudart -lmysqlclient [many other shared libraries]
gcc -shared -Wl,-soname,libA.so -o libA.so *.o [cuda_obj_file_dir]/*.o -L/usr/local/cuda/lib64 -lcudart -lmysqlclient [many other shared libraries]

Many thanks to Anton Kovalenko for advice. Unfortunately, I wasn't able to solve my ultimate goal, but maybe this will serve as an intermediate for someone else, as it is doing for me now.

Answer 2

When you link libA.so, shared libraries you use should be specified as dependencies (-L$somedirectory -lCuda). Then the dependecies become visible with ldd ./libA.so, as they should, and then LD_LIBRARY_PATH will help.

Undefined symbols are allowed in shared libraries, so it's up to you to provide enough dependencies when you're building one. If the same thing doesn't happen with libmysqlclient, it's probably because libmysqlclient is loaded due to some other dependency (or dynamically with RTLD_GLOBAL and before libA.so). Then it's probably not what you want, even though the problem is not immediately visible.

P.S.

1. You may want to read something on sonames (starting from man ld?) to decide whether you really want to link against libCuda.so (and not against libCuda.so.N as I would prefer).
2. For quick and dirty workaround (if you're still with me in this place :), try LD_PRELOAD=/full/path/to/libCuda.so your-program.