Celery Flower as Daemon

Question

Im running celery with a redis backend. I want to run celery flower as a daemon on centos 6.2.

I understand flower is a Tornado application , so I should use a process to run a tornado application as a deamon.

Normally to start flower I use this command:

celery flower --broker=redis://localhost

I read at the below link that I need to create a python script as such: http://www.charleshooper.net/blog/python-starting-tornado-apps-at-boot-using-upstart/ (Startflower.py)

import tornado.ioloop
import tornado.web
import tornado.httpserver 

if __name__ == "__main__":
    http_server = tornado.httpserver.HTTPServer(application)
    http_server.listen(port)
    tornado.ioloop.IOLoop.instance().start()

However, I am unsure what to put in the 'application' variable. I tried 'celery flower --broker=redis://localhost' and 'celery flower" but neither worked

What do i need to do to get it working as a daemon??

Answer 1

You could keep it is as a command line program and run it under the supervisord daemon. This is a common solution in the python world (although supervisord works with any command, not just python), and I use it all the time.

Supervisord makes the program think it is still running in a terminal. There are many examples how to use supervisord, but one that I use for a python proxy server can be found here, scroll down to "Installing the proxy server as a service".

Answer 2

It is preferably to run flower as a daemon using systemd. Supervisord is not compatible with Python3 that has become a new best practice. Moreover, systemd is a standard process manager for most of the modern Linux distributions.

I use systemd as a daemon for flower in Ubuntu 16.04. Though I believe the set up won't be much different for other distributions.

1. Create a systemd configuration file called, for example, flower.service. In my case, it is located in /etc/systemd/system folder. It should contain:

   [Unit]
   Description=Flower Celery Service
   
   [Service]
   User=your_user
   Group=www-data
   WorkingDirectory=/var/www/project-working-directory
   ExecStart=/home/user/miniconda3/envs/virtualenv/bin/flower --port=5555  --loglevel=info -A yourproject
   Restart=on-failure
   Type=simple
   
   [Install]
   WantedBy=multi-user.target
   

Basically, you may set all of the available options like in a terminal. By the way, you should use flower under virtual environment. Make sure that your user have privileges over a working directory.

2. Reload systemd daemon sudo systemctl daemon-reload

3. Start a flower daemon sudo systemctl start flower

That's all! This nice tutorial helped me get through the configuration process.