ubuntu ec2 - run python script at startup with arguments

Question

I have a python script i'd like to start on startup on an ubuntu ec2 instance but im running into troubles.

The script runs in a loop and takes care or exiting when its ready so i shouldn't need to start or stop it after its running.

I've read and tried a lot of approaches with various degrees of success and honestly im confused about whats the best approach. I've tried putting a shell script that starts the python script in /etc/init.d, making it executable and doing update-rc.d to try to get it to run but its failed at every stage.

here's the contents of the script ive tried:

#!/bin/bash

cd ~/Dropbox/Render\ Farm\ 1/appleseed/bin
while :
do
    python ./watchfolder18.py -t ./appleseed.cli -u ec2 ../../data/
done

i then did

sudo chmod +x /etc/init.d/script_name
sudo sudo update-rc.d /etc/init.d/script_name defaults

This doesn't seem to run on startup and i cant see why, if i run the command manually it works as expected.

I also tried adding a line to rc.local to start the script but that doesn't seem to work either

Can anybody share what they have found is the simplest way to run a python script in the background with arguments on startup of an ec2 instance.

UPDATE: ----------------------

I've since moved this code to a file called /home/ubuntu/bin/watch_folder_start

#!/bin/bash

cd /home/ubuntu/Dropbox/Render\ Farm\ 1/appleseed/bin
while :
do
    python ./watchfolder18.py -t ./appleseed.cli -u ec2 ../../data/
done

and changed my rc.local file to this:

nohup /home/ubuntu/bin/watch_folder_start &

exit 0

Which works when i manually run rc.local but wont fire on startup, i did chmod +x rc.local but that didn't change anything,

Answer 1

I found that the best solution in the end was to use 'upstart' and create a file in etc/init called myfile.conf that contained the following

description "watch folder service"
author      "Jonathan Topf"

start on startup

stop on shutdown

# Automatically Respawn:
respawn
respawn limit 99 5

script
    HOST=`hostname`
    chdir /home/ubuntu/Dropbox/Render\ Farm\ 1/appleseed/bin
    exec /usr/bin/python ./watchfolder.py -t ./appleseed.cli -u $HOST ../../data/  >> /home/ubuntu/bin/ec2_server.log 2>&1
    echo "watch_folder started"
end script

More info on using the upstart system here

http://upstart.ubuntu.com/

https://help.ubuntu.com/community/UbuntuBootupHowto

http://blog.joshsoftware.com/2012/02/14/upstart-scripts-in-ubuntu/

Answer 2

Your /etc/init.d/script_name is missing the plumbing that update-rc.d and so on use, and won't properly handle stop, start, and other init-variety commands, so...

For initial experimentation, take advantage of the /etc/init.d/rc.local script (which should be linked to by default from /etc/rc2/S99rc.local). The gets you out of having to worry about the init.d conventions and just add things to /etc/rc.local before the exit 0 at its end.

Additionally, that ~ isn't going to be defined, you'll need to use a full pathname - and furthermore the script will run as root. We'll address how to avoid this if desired in a bit. In any of these, you'll need to replace "whoeveryouare" with something more useful. Also be warned that you may need to prefix the python command with a su command and some arguments to get the process to run with the user id you might need.

You might try (in /etc/rc.local):

( if cd '/home/whoeveryouare/Dropbox/Render Farm 1/appleseed/bin' ; then
      while : ; do
           # This loop should respawn watchfolder18.py if it dies, but
           # ideally one should fix watchfolder18.py and remove this loop.
           python ./watchfolder18.py -t ./appleseed.cli -u ec2 ../../data/
      done
  else
      echo warning: could not find watchfolder 1>&2
  fi
) &

You could also put all that in a script and just call it from /etc/rc.local.

The first pass is roughly what you had, but if we assume that watchfolder18.py will arrange to avoid dying we can cut it down to:

( cd '/home/whoeveryouare/Dropbox/Render Farm 1/appleseed/bin' \
     && exec python ./watchfolder18.py -t ./appleseed.cli -u ec2 ../../data/ ) &

These aren't all that pretty, but it should let you get your daemon sorted out so you can debug it and so on, then come back to making a proper /etc/init.d or /etc/init script later. Something like this might work in /etc/init/watchfolder.conf, but I'm not yet facile enough to claim this is anything other than a rough stab at it:

# watchfolder - spawner for watchfolder18.py
description     "watchfolder program"

start on runlevel [2345]
stop on runlevel [!2345]

script
   if cd '/home/whoeveryouare/Dropbox/Render Farm 1/appleseed/bin' ; then
     exec python ./watchfolder18.py -t ./appleseed.cli -u ec2 ../../data/0
   fi
end script