I have python program (couple of scripts) that need to be run as daemon on remote machine which is CentOS 6.4. So, I thought Upstart is the way to go.
Basic requirements are:
- Easy way to start/stop daemon app
- Respawn app if it crashes
- App must run under non-privileged user
- Be able to handle SIGHUP to reload config
Upstart version on that server is 0.6.5, so stanzas setuid and setgid are not available (they appear only in 1.4). So I use official workaround from cookbook. I am able to start/stop my app, but my main problem is that my app isn't receiving reload signal.
I'll provide stripped scripts that can reproduce the issue.
Python script (app.py
):
import os
import time
import signal
import logging
logging.basicConfig(filename='hup.log', level=logging.INFO,
format="%(asctime)s: %(levelname)s: %(message)s")
log = logging.getLogger(__name__)
running = True
def run():
log.info('PROGRAM STARTUP')
log.info('Current pid: %d' % os.getpid())
while running:
log.info('Doing some hard work')
time.sleep(10)
else:
log.info('PROGRAM TEARDOWN')
def signal_handler(signum, frame):
log.info("Received Signal: %s at frame: %s" % (signum, frame))
if signum == signal.SIGTERM:
log.info('Received request to terminate daemon (SIGTERM)')
global running
running = False
elif signum == signal.SIGHUP:
log.info('Received reload config request (SIGHUP)')
pass # reload config here
signal.signal(signal.SIGHUP, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)
run()
And Upstart config (hup.conf
):
start on runlevel [2345]
stop on runlevel [!2345]
respawn
respawn limit 10 5
chdir /home/dev/prj/im
script
# activate the virtual environment
. /home/dev/.virtualenvs/myvenv/bin/activate
# This line runs script as root
# exec python app.py
# this is official workaround for running prcesses as different
# user in upstart version prior to 1.4
# Run as user 'dev'
exec su -s /bin/sh -c 'exec "$0" "$@"' dev -- python app.py
end script
Output (as you can see, actual process with python app.py
is having different PID than upstart shows):
[dev@localhost ~]$ sudo start hup
hup start/running, process 8608
[dev@localhost ~]$ ps -elf | grep app.py
F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD
4 S root 8608 1 0 80 0 - 16143 wait 19:53 ? 00:00:00 su -s /bin/sh -c exec "$0" "$@" dev -- python app.py
4 S dev 8613 8608 0 80 0 - 7866 poll_s 19:53 ? 00:00:00 python app.py
[dev@localhost ~]$ sudo reload hup
[dev@localhost ~]$ sudo stop hup
hup stop/waiting
Log shows no reload signal received:
2013-09-19 20:00:36,092: INFO: PROGRAM STARTUP
2013-09-19 20:00:36,093: INFO: Current pid: 8613
2013-09-19 20:00:36,093: INFO: Doing some hard work
2013-09-19 20:00:45,287: INFO: Received Signal: 15 at frame: <frame object at 0xba2dd0>
2013-09-19 20:00:45,287: INFO: Received request to terminate daemon (SIGTERM)
2013-09-19 20:00:45,287: INFO: PROGRAM TEARDOWN
But when I uncomment the line in hup.conf
with exec python app.py
(and comment one with exec su...
), everything works, except the app is running as root.
[dev@localhost ~]$ sudo start hup
hup start/running, process 8811
[dev@localhost ~]$ ps -elf | grep app.py
F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD
4 S root 8811 1 0 80 0 - 8412 poll_s 20:13 ? 00:00:00 python app.py
[dev@localhost ~]$ sudo reload hup
[dev@localhost ~]$ sudo stop hup
hup stop/waiting
Log file shows SIGHUP signal has been received:
2013-09-19 20:13:40,290: INFO: PROGRAM STARTUP
2013-09-19 20:13:40,290: INFO: Current pid: 8811
2013-09-19 20:13:40,291: INFO: Doing some hard work
2013-09-19 20:13:59,739: INFO: Received Signal: 1 at frame: <frame object at 0x7d44c0>
2013-09-19 20:13:59,739: INFO: Received reload config request (SIGHUP)
2013-09-19 20:13:59,739: INFO: Doing some hard work
2013-09-19 20:14:04,313: INFO: Received Signal: 15 at frame: <frame object at 0x7d44c0>
2013-09-19 20:14:04,313: INFO: Received request to terminate daemon (SIGTERM)
2013-09-19 20:14:04,313: INFO: PROGRAM TEARDOWN
My main question is how to make reload
command work when I run script under different user on this old version of upstart?