Python PyEnchant Segmentation fault or invalid pointer

Question

I have a multi-threaded Python application which calls PyEnchant. I am no Python expert and much of the code comes from a third party. What I have done is to add code to call PyEnchant from the multi-threaded application. Since I made the change the application runs for a varying length of time then crashes. I am running it on Ubuntu 12.04. I get errors like these:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffc5ffd700 (LWP 6858)]
0x00007ffff4424854 in ?? () from /usr/lib/libenchant.so.1
(gdb) backtrace
#0  0x00007ffff4424854 in ?? () from /usr/lib/libenchant.so.1
#1  0x00007ffff442517d in enchant_dict_check () from /usr/lib/libenchant.so.1
#2  0x00007ffff48a2ea4 in ffi_call_unix64 () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#3  0x00007ffff48a28c5 in ffi_call () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#4  0x00007ffff48932c2 in _ctypes_callproc () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#5  0x00007ffff4893aa2 in ?? () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#6  0x00000000004c7c76 in PyObject_Call ()
#7  0x000000000042aa4a in PyEval_EvalFrameEx ()
#8  0x000000000042abe2 in PyEval_EvalFrameEx ()
#9  0x000000000042abe2 in PyEval_EvalFrameEx ()
#10 0x00000000004317f2 in PyEval_EvalCodeEx ()
#11 0x000000000054aa40 in ?? ()
#12 0x00000000004c7c76 in PyObject_Call ()
#13 0x000000000049e9ba in ?? ()
#14 0x00000000004c7c76 in PyObject_Call ()
#15 0x00000000004c7d36 in PyEval_CallObjectWithKeywords ()
#16 0x0000000000443e3a in ?? ()
#17 0x000000000042a862 in PyEval_EvalFrameEx ()
#18 0x000000000042abe2 in PyEval_EvalFrameEx ()
#19 0x000000000042abe2 in PyEval_EvalFrameEx ()

or

*** glibc detected *** /usr/bin/python: free(): invalid pointer: 0x00007fffd4000178 ***
(gdb) backtrace
#0  0x00007ffff68fa445 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007ffff68fdbab in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007ffff6937e2e in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3  0x00007ffff6942626 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#4  0x00007ffff4424861 in ?? () from /usr/lib/libenchant.so.1
#5  0x00007ffff442517d in enchant_dict_check () from /usr/lib/libenchant.so.1
#6  0x00007ffff48a2ea4 in ffi_call_unix64 () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#7  0x00007ffff48a28c5 in ffi_call () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#8  0x00007ffff48932c2 in _ctypes_callproc () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#9  0x00007ffff4893aa2 in ?? () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#10 0x00000000004c7c76 in PyObject_Call ()
#11 0x000000000042aa4a in PyEval_EvalFrameEx ()
#12 0x000000000042abe2 in PyEval_EvalFrameEx ()
#13 0x000000000042abe2 in PyEval_EvalFrameEx ()
#14 0x00000000004317f2 in PyEval_EvalCodeEx ()
#15 0x000000000054aa40 in ?? ()
#16 0x00000000004c7c76 in PyObject_Call ()
#17 0x000000000049e9ba in ?? ()
#18 0x00000000004c7c76 in PyObject_Call ()
#19 0x00000000004c7d36 in PyEval_CallObjectWithKeywords ()
#20 0x0000000000443e3a in ?? ()
#21 0x000000000042a862 in PyEval_EvalFrameEx ()
#22 0x000000000042abe2 in PyEval_EvalFrameEx ()

or

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe57fa700 (LWP 17532)]
0x0000000000000000 in ?? ()
(gdb) backtrace
#0  0x0000000000000000 in ?? ()
#1  0x00007ffff3f5fe33 in g_hash_table_lookup () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#2  0x00007ffff44245a3 in ?? () from /usr/lib/libenchant.so.1
#3  0x00007ffff442518b in enchant_dict_check () from /usr/lib/libenchant.so.1
#4  0x00007ffff48a2ea4 in ffi_call_unix64 () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#5  0x00007ffff48a28c5 in ffi_call () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#6  0x00007ffff48932c2 in _ctypes_callproc () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#7  0x00007ffff4893aa2 in ?? () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#8  0x00000000004c7c76 in PyObject_Call ()
#9  0x000000000042aa4a in PyEval_EvalFrameEx ()
#10 0x000000000042abe2 in PyEval_EvalFrameEx ()
#11 0x000000000042abe2 in PyEval_EvalFrameEx ()
#12 0x00000000004317f2 in PyEval_EvalCodeEx ()
#13 0x000000000054aa40 in ?? ()
#14 0x00000000004c7c76 in PyObject_Call ()
#15 0x000000000049e9ba in ?? ()
#16 0x00000000004c7c76 in PyObject_Call ()
#17 0x00000000004c7d36 in PyEval_CallObjectWithKeywords ()
#18 0x0000000000443e3a in ?? ()
#19 0x000000000042a862 in PyEval_EvalFrameEx ()
#20 0x000000000042abe2 in PyEval_EvalFrameEx ()
#21 0x000000000042abe2 in PyEval_EvalFrameEx ()
#22 0x000000000042abe2 in PyEval_EvalFrameEx ()

reading another post I tried

ulimit -s unlimited

and various other values for ulimit but it did not seem to help.

I have monitored the application running with "top" and it does not seem to be using too much resources. Sometimes it runs for 5 minutes sometimes 15. During these runs it processes hundreds of strings and finds hundreds of unknown words. There are no known issues with the machine the code runs on, it is my development machine everything else runs stably.

Can anyone shed any light on what these errors mean? Is there something I should or should not be doing in my Python code which will sort it out? I guess it is a memory issue (something I know little about), should I be checking for memory leaks etc.?

Answer 1

I did some more tests and found that it works fine with a single worker thread. After a bit of Googleing I find that http://aspell.net/ says under "Things that need to be done"; "Make Aspell Thread safe. Even though Aspell itself is not multi-threaded I would like it to be thread safe so that it can be used by multi-threaded programs." I am using Aspell with PyEnchant through Enchant so I guess what I see can be expected. I contacted the author(s) of Enchant http://www.abisource.com/projects/enchant/ they said "Enchant itself is threadsafe. Unfortunately, some of its backends may not be, such as you've found with Aspell. This is something we could work around in Enchant, though, by serializing all access to the Aspell dictionary, thus hiding the thread-unsafeness to the user." and could I file a enhancement request. So I have filed an enhancement request and look forward to the enhancement in due course.