Retry Tâches perdus ou non (Céleri, Django et RabbitMQ)

Question

Y at-il un moyen de déterminer si une tâche est perdue et réessayez?

---

Je pense que la raison de la perte peut être dispatcher bug ou plantage du fil travailleur.

Je comptais les réessayer mais je ne suis pas sûr de savoir comment déterminer quelles tâches doivent être à la retraite?

Et comment faire automatiquement ce processus? Puis-je utiliser mon propre planificateur personnalisé qui permettra de créer de nouvelles tâches?

Modifier: Je trouve de la documentation RabbitMQ jamais des tâches en vrac, mais ce qui se passe quand l'accident de thread de travail au milieu de l'exécution des tâches

Answer 1

What you need is to set

CELERY_ACKS_LATE = True

Late ack means that the task messages will be acknowledged after the task has been executed, not just before, which is the default behavior. In this way if the worker crashes rabbit MQ will still have the message.

Obviously of a total crash (Rabbit + workers) at the same time there is no way of recovering the task, except if you implement a logging on task start and task end. Personally I write in a mongodb a line every time a task start and another one when the task finish (independently form the result), in this way I can know which task was interrupted by analyzing the mongo logs.

You can do it easily by overriding the methods __call__ and after_return of the celery base task class.

Following you see a piece of my code that uses a taskLogger class as context manager (with entry and exit point). The taskLogger class simply writes a line containing the task info in a mongodb instance.

def __call__(self, *args, **kwargs):
    """In celery task this function call the run method, here you can
    set some environment variable before the run of the task"""

    #Inizialize context managers    

    self.taskLogger = TaskLogger(args, kwargs)
    self.taskLogger.__enter__()

    return self.run(*args, **kwargs)

def after_return(self, status, retval, task_id, args, kwargs, einfo):
    #exit point for context managers
    self.taskLogger.__exit__(status, retval, task_id, args, kwargs, einfo)

I hope this could help