Python program possibly causing filesystem errors

Question

Overview:

I have a Python program that controls a motion sensor between the hours of 9-5. This program is running on a Raspberry Pi, stored on SD media with occidentals v2.1 Debian OS. The program relies heavily on the time.sleep method to sleep during off-hours and resume it's duties at 9 am, the next day. If the sensor triggers, it automatically takes a small picture via webcam.

Problem:

Until recently the program was working fine, until one morning when someone triggered the sensor and the take_picture() function failed to initialize /dev/video0. It seemed that udev wasn't mapping the device to /dev/video*, and when I re-plugged in the webcam and checked its status with "dmesg | tail", I received an error:

[36725.201116] EXT4-fs error (device mmcblk0p2): udev

After researching the problem and coming to the conclusion that it was filesystem corruption, I attempted to run a filesystem check on the SD media. Many errors were resolved in that process, but the same ext4-fs error messages reoccurred after re-mounting the USB webcam.

Fixing attempt #1

At this point I re-installed the OS and reran my sensor Python program; for two days it worked, sent images, and slept. I assumed that everything was OK, so I shutdown the device to place the sensor in more focused location.

After starting up the device and simply running the python command from an SSH connection, I received this error:

Traceback (most recent call last):
  File "/usr/lib/python2.7/site.py", line 562, in <module>
    main()
  File "/usr/lib/python2.7/site.py", line 544, in main
    known_paths = addusersitepackages(known_paths)
  File "/usr/lib/python2.7/site.py", line 271, in addusersitepackages
    user_site = getusersitepackages()
  File "/usr/lib/python2.7/site.py", line 246, in getusersitepackages
    user_base = getuserbase() # this will also set USER_BASE
  File "/usr/lib/python2.7/site.py", line 235, in getuserbase
    from sysconfig import get_config_var
ImportError: cannot import name get_config_var

No settings were changed between the time that the program was working and after I rebooted. I assumed the worst and checked "dmesg | tail" again and received more filesystem errors:

[36725.201116] EXT4-fs error (device mmcblk0p2): ext4_ext_check_inode:403: inode #21266: comm python: bad header/extent: invalid extent entries - magic f30a, entries 1, max 4(4), depth 0(0)
[36725.239030] EXT4-fs error (device mmcblk0p2): ext4_ext_check_inode:403: inode #21427: comm python: bad header/extent: too large eh_max - magic f30a, entries 1, max 516(4), depth 0(0)

After an additional filesystem check and reboot, running python:

[  819.427780] EXT4-fs error (device mmcblk0p2): ext4_ext_check_inode:403: inode #21266: comm python: bad header/extent: invalid extent entries - magic f30a, entries 1, max 4(4), depth 0(0)
[  819.470874] EXT4-fs error (device mmcblk0p2): ext4_ext_check_inode:403: inode #21427: comm python: bad header/extent: too large eh_max - magic f30a, entries 1, max 516(4), depth 0(0)
[  819.570893] EXT4-fs error (device mmcblk0p2): ext4_xattr_block_get:232: inode #21265: comm python: bad block 128

At this point I am thinking that the reason for these errors might have something to do with the core of my program, but in essence it is quite simple and only writes to disk upon creating pictures. Below is the culprit code:

Code with potential issue:

def take_picture():
    cam = pygame.camera.Camera("/dev/video0",(320,240))     #1280x960 max but slow
    pygame.init()
    cam.start()
    print "Capturing Image..."
    image_output = cam.get_image()
    pygame.image.save(image_output,'img_' +str(get_timestamp)+'.jpg')
    cam.stop()

# Activate PIR
def pir_init():
    while True:
        m = datetime.datetime.now().time().minute     #loop through for current time
        h = datetime.datetime.now().time().hour
        d = datetime.datetime.today().weekday()
        if d > 4:
            print("Sleeping...")
            print time_sleep
            time_sleep = (((7 - d) * 3600) * 24) - (h * 3600) + (9 * 3600)
            time.sleep(time_sleep)
        elif h < 9:
            print("Sleeping...")
            time_sleep = (9 - h) * 3600
            time.sleep(time_sleep)
        elif h > 16:
            print("Sleeping...")
            time_sleep = ((24 - h) + 9) * 3600
            time.sleep(time_sleep)
        elif io.input(pir_pin):
            print("PIR ALARM")
            pygame.camera.init()
            time.sleep(1.5)
            take_picture()
            send_gmail()
            time.sleep(10)      # Cooldown to prevent email spew
        else:
            print "Waiting..."
            time.sleep(0.5)     # Sleep and wait for movement
            print m
print "Wait"

The only potential issue I can see above might be with generating the picture or sleeping for an extended amount of time.

What might be causing these repeated filesystem errors?

Answer 1

It is pretty common for SD cards to become corrupt on the Raspberry Pi and this code would not directly be able to cause this.

The only reason I think that the code would be causing problems that I see is if it tries to write to the SD card too often, which is unlikely and most likely caused by an underlying issue. Maybe the SD card is going bad, or came from a bad batch.

If you are simply sending these as an email it might be an alternative to not store the pictures on your hard-drive at all. Or you could add an USB hard-drive or memory stick and use that to store the pictures.

You could also cache the pictures and write out a couple at a time instead of writing each one out directly to reduce the load on the SD card. This would be as simple as storing let’s say five pictures in memory, and once you have taken five pictures, write them out.

pictures = {}
pictures[get_timestamp] = image_output

Then once you have reached a total of five entries you loop through them, use the key to generate the file-name while the value contains the actual picture. You then save all five on your hard-drive.

If you have the 512 MB version you could increase the number of pictures stored in memory, but keep in mind that in case of a power failure you would lose anything in memory.