Can I run pika's connection.ioloop.start() in a daemon?

Question

I'm trying to set up a worker daemon for processing messages from rabbitmq. I'm using pika and its SelectConnection. The code works fine if I don't run it as a daemon. I can use

py worker.py

and

py worker.py &

successfully. However, when I add

import daemon
with daemon.DaemonContext():
    connection.ioloop.start()

to worker.py, The code, while it doesn't raise any exceptions, stops fetching messages from the queue and maxes out my CPU utilization. worker.py looks exactly like this example.

thanks.

Answer 1

First, I have no idea where you got this daemon module, but I noticed that you import daemon in your example but not pika.

import daemon
with daemon.DaemonContext():
    import pika
    # any additional pika code here, maybe the creation of connection?
    connection.ioloop.start()

I strongly suspect that you use pika (even import executes code inside the module) before you turn the process into a daemon. This would mean that you have resources in RAM that are copied when the daemon forks, and if this daemon module does the right thing, then any open sockets or files will be forced to close during the process.

It is better to not have any objects whose lifetime spans the daemon boundary. Make daemons run entirely separate from any other processing, and have them communicate with other processes via message passing such as AMQP or 0MQ.