pysqlite segmentation fault?

Question

I am on OSX 10.6 and have recently upgraded my Python from 2.6 to 2.7, so I had to upgrade python packages.

This time I decided to go with brew and installed sqlite, libspatialite and spatialite-tools with brew and brew doctor says everything is OK. At that time when I start my local development server (Django 1.4), it was complaining that the existing pysqlite does not support extension loading (which is required by SpatiaLite).

Then I downloaded pysqlite-2.6.3, unpacked, make the config change to enable extension loading, then did:

python setup.py build_static
python setup.py install

as described here.

When I run the dev server, now I am getting a "Segmentation Fault". As this doesn't tell much, I added settrace to Django's manage.py just after import statements:

def trace(frame, event, arg):
    print "%s, %s:%d" % (event, frame.f_code.co_filename, frame.f_lineno)
    return trace

sys.settrace(trace)

The few lines before the Segmentation Fault are as follows:

...
call, /Users/omat/workspace/devspaces/env/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py:71
line, /Users/omat/workspace/devspaces/env/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py:71
call, /Users/omat/workspace/devspaces/env/lib/python2.7/encodings/utf_8.py:15
line, /Users/omat/workspace/devspaces/env/lib/python2.7/encodings/utf_8.py:16
return, /Users/omat/workspace/devspaces/env/lib/python2.7/encodings/utf_8.py:16
return, /Users/omat/workspace/devspaces/env/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py:71
Segmentation fault

Any ideas on what might be wrong and some help is much appreciated. Thanks.

Answer 1

Most likely you have a non-compatible SQLite or Python SQLite native bindings libraries mixed with your Python.

Reinstall all packages in the question after Python upgrade.

If you want to avoid problems like this altogether my suggestion would install Python, SQLite and Python bindings using a managed environment where all the packages come from the same source, like from Homebrew.

If you manually install packages outside this environment then make sure a correct Python libraries and Python headers are used when compiling the native libraries. I.e if you use libraries from Homebrew, use Homebrew supplied Python and Python headers, not OSX defaults.

How to trace segfaults in Python

http://wiki.python.org/moin/DebuggingWithGdb

This will tell you the actual problem and the individual library in the question which is failing.

Answer 2

I have recently (2018) encountered this problem using a newly created Python virtualenv in various system environments.

The cause of the segment fault was initially isolated to SQLite by installing Python package faulthander, and modifying my site's manage.py to activate it, thus:

import faulthandler
faulthandler.enable()

My first encounter with the problem was on MacOS using an older version of Python (I forgot which, maybe about 2.7.12). On that occasion, the problem was resolved by upgrading to Python 2.7.15 and regenerating the virtualenv.

Next was on Ubuntu 14.04 running Python 2.7.5. Thinking this was due to an old version of Ubuntu I upgraded to 16.04 and Python 2.7.12, but the problem persisted. On this occasion, I wanted to stick with the standard distribution install of Python.

This Django ticket suggests a problem with an older version of SQLite (which I surmise ships with older Python distributions), and also contains a link to this Gist that explains how to upgrade SQLite for Python to a working version.

So, in summary, on Ubuntu 16.04 with Python 2.7.12:

1. Install prerequisite packages:

   sudo apt-get install python-dev
   sudo apt-get install libsqlite3-dev
   

2. Create and activate a new Python virtualenv.

3. Create and go to a working directory; e.g.

   mkdir ~/sqlite
   cd ~/sqlite
   

4. Run the script described in this Gist; i.e.

   # install custom sqlite 3.14
   wget https://github.com/ghaering/pysqlite/archive/2.8.3.tar.gz
   wget https://www.sqlite.org/2016/sqlite-autoconf-3140100.tar.gz
   tar -xzvf sqlite-autoconf-3140100.tar.gz
   tar -xzvf 2.8.3.tar.gz
   cp -av sqlite-autoconf-3140100/. pysqlite-2.8.3/
   cd ./pysqlite-2.8.3 && python setup.py build_static install
   

At this point, I was able to install my software and runs tests without any problems (well, without any problems not of my own making :-o).