Question

Prefacing my question with the fact that I'm new to Celery and this (1) may have been answered somewhere else (if so, I couldn't find the answer) or (2) there may be a better way to accomplish my objective than what I'm directly asking.

Also, I am aware of celery.contrib.methods, but task_method does not quite accomplish what I am looking for.

My Objective

I would like to create a class mixin that turns a whole class into a Celery task. For example, a mixin represented by something like the code below (which right now does not run):

from celery import Task

class ClassTaskMixin(Task):

    @classmethod
    def enqueue(cls, *args, **kwargs):
        cls.delay(*args, **kwargs)

    def run(self, *args, **kwargs):
        Obj = type(self.name, (), {})
        Obj(*args, **kwargs).run_now()

    def run_now(self):
        raise NotImplementedError()

Unlike when using task_method, I do not want to fully instantiate the class before the task is queued and .delay() is called. Rather, I want to simply hand-off the class name along with any relevant initialization parameters to the async process. The async process would then fully instantiate the class using the class name and the given initialization paremeters, and then call some method (say .run_now(), for example) on the instantiated object.

Example Use Case

Constructing and sending email asynchronously would be an example use for the mixin I need.

class WelcomeEmail(EmailBase, ClassTaskMixin):

    def __init__(self, recipient_address, template_name, template_context):
        self.recipient_address = recipient_address
        self.template_name = template_name
        self.template_context = template_context

   def send(self):
       self.render_templates()
       self.construct_mime()
       self.archive_to_db()
       self.send_smtp_email()

   def run_now(self):
       self.send()

The above code would send an email in an async Celery process by calling WelcomeEmail.enqueue(recipient_address, template_name, template_context). Sending the email synchronously in-process would be accomplished by calling WelcomeEmail(recipient_address, template_name, template_context).send().

Questions

  1. Is there any reason that what I'm trying to do is very, very wrong within the Celery framework?
  2. Is there a better way to structure the mixin to make it more Celery-onic than what I've proposed (better attribute names, different method structure, etc.)?
  3. What am I missing to make the mixin functional in a use case as I've described?
Was it helpful?

Solution

Apparently this issue isn't hugely interesting to a lot of people, but... I've accomplished what I set out to do.

See pull request https://github.com/celery/celery/pull/1897 for details.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top