Celery returns no results when using node-celery(MeteorJS) and ampq backend

Question

I just started using Celery, where a Celery worker is written in Python, and the tasks are sent from node/Meteor using node-celery.

Why is there no result returned from the client.call()? Python worker console shows that the task has been sent and successfully processed. But none of the events ready and pending seems to be firing!

Using Celery 3.1.7, RabbitMQ 3.2.2, node-celery 0.1.1, Meteor 0.7.0.1

Node

var celery = Meteor.require('node-celery'),
    client = celery.createClient({
        CELERY_BROKER_URL: 'amqp://guest:guest@localhost:5672//',
        CELERY_RESULT_BACKEND: 'amqp://',
        CELERY_TASK_SERIALIZER: 'json',
        CELERY_RESULT_SERIALIZER: 'json'
    });

client.on('error', function(err) {
    console.log(err);
});

client.on('connect', function() {
    console.log('Connected')

    var results = client.call('tasks.echo', ['Hello world'], function(result) {
        console.log('results:' + result);
    });

    results.on('pending', function(result) {
        console.log('pending: ' + result)
    });

    results.on('ready', function(result) {
        console.log(result)
    });
});

Output

Connected

Answer 1

Edit:

You now can find my celery package on atmosphere, or just install with mrt install celery

---

I've been through the pain of setting this up, but have come out the other side.

I endend up having to branch node-celery (https://github.com/nathan-muir/node-celery), and node-amqp (https://github.com/nathan-muir/node-amqp) to create workable versions.

I have bottled all this in to a meteor package - but I'm yet to clean everything up for general consumption. (https://github.com/nathan-muir/meteor-celery). I'd like to wrap it with Fibers/Futures instead of leaving it callback style (pull requests welcome.

The package reads from Meteor.settings.celery, so create a settings file like:

{
  "celery": {
    "CELERY_BROKER_URL": "amqp://guest@localhost:5672//",
    "CELERY_RESULT_BACKEND": "amqp",
    "CELERY_SEND_TASK_SENT_EVENT": true
  }
}

And start meteor with meteor --settings path/to/settings.json

I also run my client & workers with events enabled (celery worker -E --config=xx), so I can use the celery monitoring tools (and my own custom cloudwatch stats monitor https://github.com/nathan-muir/celery-cloudwatch)

If you've got any more questions - feel free to ask in the comments.

Answer 2

I'm not using Meteor but I was having a similar issue with node-celery. Hopefully this will help someone else that will stumble on this question.

In my case I was able solve it by adding the following in the celeryconfig.py file

CELERY_TASK_SERIALIZER = 'json'

CELERY_ACCEPT_CONTENT = ['json']

CELERY_RESULT_BACKEND = 'amqp'

CELERY_RESULT_SERIALIZER = 'json'

My node-celery call from nodejs looked like this:

var client = celery.createClient({ CELERY_BROKER_URL: photobutikConfig.tracer.brokerURL,
                                   CELERY_RESULT_BACKEND: 'amqp',
                                CELERY_RESULT_SERIALIZER: 'json'}),
        tracerRequestType = ['tracer.tasks', requestType].join('.');

    client.on('connect', function() {
        var result = client.call(tracerRequestType, [payload]);
        result.on('ready', function (result) {
            client.broker.destroy();
        });
    });