Upstart - how to send reload signal to script started as different user?

Question

I have python program (couple of scripts) that need to be run as daemon on remote machine which is CentOS 6.4. So, I thought Upstart is the way to go.

Basic requirements are:

• Easy way to start/stop daemon app
• Respawn app if it crashes
• App must run under non-privileged user
• Be able to handle SIGHUP to reload config

Upstart version on that server is 0.6.5, so stanzas setuid and setgid are not available (they appear only in 1.4). So I use official workaround from cookbook. I am able to start/stop my app, but my main problem is that my app isn't receiving reload signal.

I'll provide stripped scripts that can reproduce the issue.

Python script (app.py):

import os
import time
import signal
import logging

logging.basicConfig(filename='hup.log', level=logging.INFO,
    format="%(asctime)s: %(levelname)s: %(message)s")
log = logging.getLogger(__name__)


running = True

def run():
    log.info('PROGRAM STARTUP')
    log.info('Current pid: %d' % os.getpid())

    while running:
        log.info('Doing some hard work')
        time.sleep(10)
    else:
        log.info('PROGRAM TEARDOWN')

def signal_handler(signum, frame):
    log.info("Received Signal: %s at frame: %s" % (signum, frame))

    if signum == signal.SIGTERM:
        log.info('Received request to terminate daemon (SIGTERM)')
        global running
        running = False
    elif signum == signal.SIGHUP:
        log.info('Received reload config request (SIGHUP)')
        pass  # reload config here


signal.signal(signal.SIGHUP, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)

run()

And Upstart config (hup.conf):

start on runlevel [2345]
stop on runlevel [!2345]

respawn
respawn limit 10 5

chdir /home/dev/prj/im

script
    # activate the virtual environment
    . /home/dev/.virtualenvs/myvenv/bin/activate

    # This line runs script as root
    # exec python app.py

    # this is official workaround for running prcesses as different
    # user in upstart version prior to 1.4
    # Run as user 'dev'
    exec su -s /bin/sh -c 'exec "$0" "$@"' dev -- python app.py
end script

Output (as you can see, actual process with python app.py is having different PID than upstart shows):

[dev@localhost ~]$ sudo start hup
hup start/running, process 8608
[dev@localhost ~]$ ps -elf | grep app.py
F S UID        PID  PPID  C PRI  NI ADDR SZ WCHAN  STIME TTY          TIME CMD
4 S root      8608     1  0  80   0 - 16143 wait   19:53 ?        00:00:00 su -s /bin/sh -c exec "$0" "$@" dev -- python app.py
4 S dev       8613  8608  0  80   0 -  7866 poll_s 19:53 ?        00:00:00 python app.py
[dev@localhost ~]$ sudo reload hup
[dev@localhost ~]$ sudo stop hup
hup stop/waiting

Log shows no reload signal received:

2013-09-19 20:00:36,092: INFO: PROGRAM STARTUP
2013-09-19 20:00:36,093: INFO: Current pid: 8613
2013-09-19 20:00:36,093: INFO: Doing some hard work
2013-09-19 20:00:45,287: INFO: Received Signal: 15 at frame: <frame object at 0xba2dd0>
2013-09-19 20:00:45,287: INFO: Received request to terminate daemon (SIGTERM)
2013-09-19 20:00:45,287: INFO: PROGRAM TEARDOWN

But when I uncomment the line in hup.conf with exec python app.py (and comment one with exec su...), everything works, except the app is running as root.

[dev@localhost ~]$ sudo start hup
hup start/running, process 8811
[dev@localhost ~]$ ps -elf | grep app.py
F S UID        PID  PPID  C PRI  NI ADDR SZ WCHAN  STIME TTY          TIME CMD
4 S root      8811     1  0  80   0 -  8412 poll_s 20:13 ?        00:00:00 python app.py
[dev@localhost ~]$ sudo reload hup
[dev@localhost ~]$ sudo stop hup
hup stop/waiting

Log file shows SIGHUP signal has been received:

2013-09-19 20:13:40,290: INFO: PROGRAM STARTUP
2013-09-19 20:13:40,290: INFO: Current pid: 8811
2013-09-19 20:13:40,291: INFO: Doing some hard work
2013-09-19 20:13:59,739: INFO: Received Signal: 1 at frame: <frame object at 0x7d44c0>
2013-09-19 20:13:59,739: INFO: Received reload config request (SIGHUP)
2013-09-19 20:13:59,739: INFO: Doing some hard work
2013-09-19 20:14:04,313: INFO: Received Signal: 15 at frame: <frame object at 0x7d44c0>
2013-09-19 20:14:04,313: INFO: Received request to terminate daemon (SIGTERM)
2013-09-19 20:14:04,313: INFO: PROGRAM TEARDOWN

My main question is how to make reload command work when I run script under different user on this old version of upstart?

Answer 1

I ended up running my script as root and then dropping privileges to desired user. To run script as root I've uncommented exec python app.py from Upstart config above. To drop privileges I use slightly modified code from this answer:

def drop_privileges(uid_name, gid_name):
    '''If running as root, this function will try to change current uid and gid
    to given values.
    May raise OSError in case of error.
    :param uid_name: Name of the user we need to be running as.
    :param gid_name: Name of the group we need to be running as.
    '''

    starting_uid = os.getuid()
    starting_gid = os.getgid()

    log.info('drop_privileges: started as %s/%s' %
        (pwd.getpwuid(starting_uid).pw_name,
        grp.getgrgid(starting_gid).gr_name))

    if starting_uid == 0:
        # If we started as root, drop privs and become the specified user/group
        log.info("drop_privileges: trying to drop provileges to '%s'" % uid_name)

        # Get the uid/gid from the name
        running_uid = pwd.getpwnam(uid_name).pw_uid
        running_gid = grp.getgrnam(gid_name).gr_gid

        # Try setting the new uid/gid
        # These calls may result in exception, let it propagate back
        # Exception thrown is: OSError (e.errno == 1 means 'Operation not
        # permitted')
        os.setgid(running_gid)
        os.setuid(running_uid)

        final_uid = os.getuid()
        final_gid = os.getgid()
        log.info('drop_privileges: running as %s/%s' %
            (pwd.getpwuid(final_uid).pw_name,
             grp.getgrgid(final_gid).gr_name))