Django and fcgi and mod_fcgid errors

Question

Ok I'm exhausted with this problem on my shared server at Bluehost and I haven't been able to find any ideas.

---

I've uploaded my app and everything appears to be working fine except for one thing, the photo uploader that I'm using (which is the core of the site). I'm using the stdimage2 Django app (a fork of stdimage) and it worked fine on the development server (Ubuntu 10.04). But it's having major problems at the moment and I'm looking for any possible ideas or inklings of things that I'm missing.

Here's the errors that are being printed out (this is with four pictures being sent as separate requests one after the other by the uploader):

[Wed Dec 21 09:47:03 2011] [warn] [client 128.187.97.6] (104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server, referer: http://gallery.fernstenfamily.com/backend/upload/1/
[Wed Dec 21 09:47:03 2011] [error] [client 128.187.97.6] Premature end of script headers: mySite.fcgi, referer: http://gallery.fernstenfamily.com/backend/upload/1/
[Wed Dec 21 09:47:03 2011] [warn] RewriteCond: NoCase option for non-regex pattern '-f' is not supported and will be ignored.
[Wed Dec 21 09:47:03 2011] [warn] [client 128.187.97.6] (104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server, referer: http://gallery.fernstenfamily.com/backend/upload/1/
[Wed Dec 21 09:47:03 2011] [error] [client 128.187.97.6] Premature end of script headers: mySite.fcgi, referer: http://gallery.fernstenfamily.com/backend/upload/1/
[Wed Dec 21 09:47:04 2011] [warn] [client 128.187.97.6] (104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server, referer: http://gallery.fernstenfamily.com/backend/upload/1/
[Wed Dec 21 09:47:04 2011] [error] [client 128.187.97.6] Premature end of script headers: mySite.fcgi, referer: http://gallery.fernstenfamily.com/backend/upload/1/
[Wed Dec 21 09:47:05 2011] [error] mod_fcgid: process /home3/fernsten/public_html/ffamily/gallery/mySite.fcgi(22492) exit(communication error), get unexpected signal 11

This is what my fcgi file looks like:

#!/home3/fernsten/local/Python-2.7/bin/python
import sys, os
print "sys.path is ", sys.path

#add a custom Python path and pray it works dddd
sys.path.insert(0, "/home3/fernsten/local/lib/python2.7/site-packages")
sys.path.insert(0, "/home3/fernsten/local/lib/python2.7")
sys.path.insert(0, "/home3/fernsten/local/lib/python2.7/site-packages/flup-1.0.2-py2.7.egg")
sys.path.insert(0, "/home3/fernsten/django_projects")
sys.path.insert(0, "/home3/fernsten/django_projects/gallery")
sys.path.insert(0, "/home3/fernsten/local/lib/python2.7/site-packages/MySQL_python-1.2.3-   py2.7-linux-x86_64.egg")

os.chdir("/home3/fernsten/django_projects/gallery")

os.environ['DJANGO_SETTINGS_MODULE'] = "gallery.settings"

from django.core.servers.fastcgi import runfastcgi
runfastcgi(method="threaded", daemonize="false")

And lastly here's the function where it dies:

if request.method == 'POST':
    result = []
    newform = Photo()
    newform.image= request.FILES.get('file')
    newform.album=Album.objects.get(pk=album_pk)
    newform.save()
    result.append({"name":"Uploaded image",
                    "size":newform.image.size,
                    "url":MEDIA_URL + newform.image.url,
                    "thumbnail_url": newform.image.url.replace(".",".thumbnail."),
                    "delete_url":'/backend/delete_img/' + str(newform.pk)+'/',
                    "delete_type":"POST",
                    })
    response_data = simplejson.dumps(result)
    return HttpResponse(response_data, mimetype='application/json')
return render_to_response('backend/upload_images.html',{'album_pk':album_pk},context_instance=RequestContext(request))

I'm fairly certain it's happening in this area because when I do newform.save() stdimage2 does things in the background, such as renaming the uploaded picture and creating and naming a thumbnail. The image is being renamed but no thumbnail is being generated.

Any ideas would be awesome as I haven't been able to find much about what could be wrong.

---

UPDATE

I also cannot upload the image using the admin interface in Django. Well I can upload the image but it breaks at that point. StdImage2 changes the uploaded images name per the model definition but doesn't create the thumbnail. In the database it's storing the URL to the image with the uploaded image name, not the generated one.

To cover all my bases I copied the files from the server and ran it locally without any problems at all. Which has only caused more confusion and frustration.

---

UPDATE 2

After using the Django shell and trying it that way (at least I hope that I did it correctly I get the following error:

image = Image.open('/home3/fernsten/www/ffamily/media/media/albums/1/image_1.jpeg')
>>> newform.image = image
>>> newform.album = Album.objects.get(pk=1)
>>> newform.save()
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home3/fernsten/local/lib/python2.7/site-packages/django/db/models/base.py", line 460, in save self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/home3/fernsten/local/lib/python2.7/site-packages/django/db/models/base.py", line 543, in save_base for f in meta.local_fields if not isinstance(f, AutoField)]
File "/home3/fernsten/local/lib/python2.7/site-packages/django/db/models/fields/files.py", line 253, in pre_save
if file and not file._committed:
File "/home3/fernsten/local/lib/python2.7/site-packages/PIL/Image.py", line 512, in   __getattr__
raise AttributeError(name)
AttributeError: _committed

Answer 1

There appears to be a segmentation fault occurring somewhere -- according to the last line in your error log. This shouldn't happen with Python code, so I'd start by looking at the code to stdimage2 to see what else is in there.

A quick perusal of the source suggests that it's all Python as well, and that the only external library that it depends on is PIL. PIL is a C library (at least part of it), so it's possible that that is where the error is occurring, especially if, as you say, the fields are renamed, but the resized images are not being created.

How was PIL installed on the server? If it was installed through a package manager, or compiled right on the server itself, then it might be necessary to update it, or recompile. If you copied the module from your development machine, then there may be a conflict there (different libraries, different processor architecture, etc.)

The other way I would go about troubleshooting this is from a python shell, rather than through the web interface. Try to load your model and trigger the save() that seems to be failing. If the shell terminates with a segmentation fault, then try it again, but this time single-stepping with a debugger. (I would actually put a pdb breakpoint on StdImageField._resize_image, and run the code until it hit that breakpoint.)

I don't think there's an easier way to debug this; there doesn't seem to be anything obvious in your code that would cause it, so it's looking like something on the server itself.