h5py causing deadlock when used together with another HDF5 module

Question

I am writing a C++ module for Python that uses pthreads and HDF5. My module creates HDF5 files in H5F_ACC_EXCL mode so that it fails when a file is already present. HDF5 prints a stack trace in this case. The python C interface functions for this module are executed in one thread and the file creation happens in another one. Normally all works fine, but if I import h5py in a script using my module, the system gets into a deadlock. The thread that tries to create the file has the following backtrace:

#0  sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:86
#1  0x00000000004aed31 in PyThread_acquire_lock ()
#2  0x0000000000495c74 in PyEval_RestoreThread ()
#3  0x00000000004fd83a in PyGILState_Ensure ()
#4  0x00007ffff02ca907 in __pyx_f_4h5py_3h5e_err_callback (__pyx_v_client_data=0xc5c470) at h5py/h5e.c:2045
#5  0x00007ffff5be1d62 in H5E_dump_api_stack () from /usr/lib/libhdf5.so.6
#6  0x00007ffff5be6a98 in H5Fcreate () from /usr/lib/libhdf5.so.6
#7  0x00007fffecb0cc46 in HDF5WriterBase::openFile (this=0x7fffe47efd68) at HDF5WriterBase.cpp:143

[Line #7 informs about the function in my module] I found that importing PyTables does not cause this problem. Any help on fixing this issue will be much appreciated.

Answer 1

This may not be related, but I had some strange issues while trying to use h5py and libjhdf5 (java code was being called using jpype). Here are some details of the bug: https://bugs.launchpad.net/openquake/+bug/882637

That may not be directly helpful, but I will point out that this particular bug was only reproducible using a specific version of libhdf5. I had this problem on Ubuntu 11.04 using default h5py, libjhdf, and libhdf packages. One of my colleagues was using using a different distro (and a different package) and he did not have this issue.

Long story short, what os, version, and packages are you using? It's possible the package has a bug. Code samples to reproduce the error would also be helpful.