Using GHC with NVCC

Question

As an alternative to accelerate, I'm trying to call CUDA code over Haskell's FFI.

Here's a simple program that fails to compile:

cuda_code.cu:

void cuda_init() {
    cudaFree (0);
    cudaThreadSynchronize ();
}

Test.hs:

foreign import ccall unsafe "cuda_init" cuda_init :: IO ()
main = cuda_init

I compiled with

$> nvcc -c -o cuda_code.o cuda_code.cu
$> ghc Test cuda_code.o

and got several linking errors (undefined reference to cudaFree, etc). This isn't terribly surprising, and the obvious solution (to me) is to link with NVCC using -pgml nvcc. (This worked when I was using Intel CILK+ in my C code: I simply changed the linker to ICC, and everything worked just fine.)

Howver, using NVCC to link results in the linking error:

ghc Test -pgml nvcc cuda_code.o
[1 of 1] Compiling Main             ( Test.hs, Test.o )
Linking Test ...
nvcc fatal   : Unknown option 'u'

Running

strace -v -f -e execve ghc Test -pgml nvcc cuda_code.o

(is there an easier way?) I discovered ghc is calling nvcc with

nvcc ... -L~/ghc... -L... -l... -l... -u ghczmprim_GHC... -u ghc...

I assume the -u options are intended for linking gcc (and apparently icc) with undefined symbols, something nvcc clearly doesn't like.

I have no knowledge about how GHC links files. Thoughts on how I can get GHC to link to my CUDA code?

--------EDIT-----------------

Someone suggested that I try linking with GCC (as usual), but pass in the necessary linker options to gcc so that it can link to the CUDA libraries. If anyone knows what these might be, this would probably work!

Answer 1

I figured out how to make this work.

cudaTest.cu:

// the `extern "C"` is important! It tells nvcc to not 
// mangle the name, since nvcc assumes C++ code by default
extern "C" 
void cudafunc() {
  cudaFree(0);
  cudaThreadSynchronize();
}

Test.hs

foreign import ccall unsafe "cudafunc" cudaFunc :: IO ()
main = cudaFunc

Compile with:

>nvcc -c -o cudaTest.o cudaTest.cu
>ghc --make Test.hs -o Test cudaTest.o -optl-lcudart

I also tried giving GHC the option -pgmc g++ and removing the extern "C" (which I expected to work), but got compile errors in some CUDA header files. There's probably some easy way to fix this so that you don't need to explicitly tag every function with extern "C".

Answer 2

GHC uses /usr/lib/ghc/settings to determine compiler and linker options, and per-package files like /var/lib/ghc/package.conf.d/builtin_rts.conf to determine package-specific linker options. (Custom directory installation will have them in ${GHC}/lib/ghc-${VERSION}/settings and ${GHC}/lib/ghc-${VERSION}/package.conf.d respectively.)

Here is what I found for the RTS:

ld-options: -u ghczmprim_GHCziTypes_Izh_static_info -u
            ghczmprim_GHCziTypes_Czh_static_info -u
            ghczmprim_GHCziTypes_Fzh_static_info -u
            ghczmprim_GHCziTypes_Dzh_static_info
            ...

According to the ld man page, the -u option defines a symbol as an undefined extern that must be defined somewhere else.

As far as I know this is the ONLY package that has these custom -u options in the ld-options: section of package.conf.d.

These must be unfortunately translated for a compiler/linker that uses a different option interface.

Be so kind and keep people posted about it on haskell-cafe@haskell.org. I'm sure there are others trying something like this too!