Crontab / Launchd: OS X User Permissions, Environment Variables, Python Virtualenvs

Question

I am trying to automate running a python script every minute on my mac, from a virtual environment. I am convinced that I don't properly understand permissions, paths, and environment variables in some crucial way that is preventing me from figuring this out.

I am an admin user with root rights enabled. I run HomeBrew, PIP and Virtualenv to manage python packages and virtual environments for different projects.

I would like to do the following every 60 seconds:

$ source /.virtualenvs/myenvironment/bin/activate
$ cd ~/desktop/python/
$ python myscript.py
$ deactivate  

I have tried:

(a) writing my own plist for Launchd - and I believe these documents were well formed.

(b) programs which manage Launchd daemons and agents for you (both Launch Control and Lingon).

(c) I have tried simply editing the crontab (only lets me if I use the sudo command).

The python script, which works on command, pulls data from an online source and stores it in a sqlite table. I can tell the cron isn't running because the sqlite db isn't being touched.

Any thoughts would be enormously appreciated.

Answer 1

You don't say exactly what you tried with launchd and cron, but I'd bet you weren't using either of them correctly. Both are oriented toward running single, isolated commands (/programs), not sequences of shell commands. While it's possible to do this with a single cron job or launchd item, it's going to be messy. The simplest thing would be to write a shell script that does the sequence you want (be sure to include a shebang at the beginning, and enable execute permission on the script with chmod +x /path/to/script), and run that from either cron or launchd:

#!/bin/bash
source /.virtualenvs/myenvironment/bin/activate
cd ~/desktop/python/
python myscript.py
deactivate

I would not recommend using Automator to wrap the command sequence; it's designed for GUI-based scripting, and may not work right in a background-only job.

Answer 2

I have had this exact same problem and have recently solved it. Look here for the steps I took. Basically it deals with the shell needing the PYTHONPATH, not just the PATH.

Answer 3

So I'm still not sure why I can't configure Launchd or the Crontab to work for me: I still suspect it has to do with the environment variables.

If anyone else is having a tough time with this issue, I suggest creating an Automator application that runs the shell script you want to run, then use Lingon or Launch Control to run the Automator application file at whatever frequency you desire.