PyOpenCl: how to debug segmentation fault?

Question

I have PyOpenCL code with OpenCL C kernel code. I catch segmentation fault error when I run my app. How to debug such error with some debugger or some other development tool? I don't know what exactly to do to find out the problem. I have in mind option with printf or something but I want use more powerful stuff.

I believe that error in kernel code, so I want to debug kernel code firstly.

UPD. I'm on linux (Arch Linux, 3.6.11), python 2 or 3, PyOpenCl 2012.1

Answer 1

Kernel debugging is an implementation-dependent affair. On Linux, the best I've found is to use AMD's CL implementation on the CPU, compile the kernel with -g, and use gdb. They've got instructions on this in their programming guide, here:

AMD CL Documentation page

Answer 2

If you're using nvidia instead of ATI/AMD GPU, the OpenCL support in nvidia SDK is...less than desired.

Intel provides a CPU-based OpenCL SDK for their recent processors, see http://software.intel.com/en-us/vcsource/tools/opencl-sdk-2013 -- (to use the RPM packages they provide on Ubuntu, you need to run "fakeroot alien --to-deb" on each package, then "dpkg -i").

With that SDK, you need to add the "-g" and "-s filename" flags to the compiler options in build(). (If your kernel only exists as a string in your program, you can add code to save it to a file just before you run it.) Then try "gdb --args python-cmd", you can start debugging by typing "break mykernel", answer Y when asked if you want to wait for the "mykernel" symbol to be dynamically loaded, then type "run".

Once you have the debugger running manually typing the command, I suggest making an executable shell script to launch your favorite .py file with the debugger (which will also be a convenient place to add hackery to your application's launch, e.g. python -m unittest, PYTHONPATH, virtualenv, LD_LIBRARY_PATH, LD_PRELOAD, etc.).

Answer 3

I wouldn't jump into conclusions without fully testing your software suite. You are running the last released version of PyOpenCl. Chances are you are passing in something to the module that isn't being populated correctly and the backend module doesn't do the necessary error-checking before using something that isn't properly populated (impossible to really help you debug without any code being made available)

Have you tried using the python debugger to set different breakpoints (import pdb; pdb.set_trace()) right before different pyopencl calls to even see where in your code it is seg faulting? This should definitely be your first task. When you find out where it is seg faulting, you need to closely look at pyopencl examples/api to see why you've errored out.

Answer 4

Segmentation fault is usually because of a bad memory access in your kernel. There is a cool tool to detect bad memory accesses, similar to valgrind: https://github.com/jrprice/Oclgrind . In combination with some printfs in the kernel code it makes it quite a bit easier to localize the problem.