How to process scheduled, recurring jobs with Kue?

Question

In my webapp, users can create recurring invoices that need to be generated and sent out at certain dates each month. For example, an invoice might need to be sent out on the 5th of every month.

I am using Kue to process all my background jobs so I want to do it in this case as well.

My current solution is to use setInterval() to create a processRecurringInvoices job every hour. This job will then find all recurring invoices from database and create a separate generateInvoice job for each recurring invoice.

The generateInvoice job will then actually generate the invoice, and if needed, will also in turn create a sendInvoiceToEmail job that will email the invoice.

At the moment this solution looks good to me, because it has a nice separation of concerns, however, I have the following questions:

1. I am not sure if I should wait for all the 'child' jobs to complete before I call done() on the main processRecurringInvoices job?
2. Where should I handle errors? Should i pass them back to the processRecurringInvoices job or should I handle them separately for each job?
3. How can I make sure that if processing takes extra long time (more than an hour), and either processRecurringInvoices or any of the child jobs are still runnning, the processRecurringInvoices job is not created again? Kind of like a unique job, or mutual exclusion?

Answer 1

1. Instead of "processRecurringInvoices" it might be easier to think of it as a job that initiates other, separate invoice-processing jobs. Thinking of it this way, once the invoice processing jobs have been enqueued, you can safely call done() on the job that kicks them all off.
2. Thinking of the problem in the way described in question 1, errors should be handled within each of the individual invoice processing jobs. If an error occurred finding potential invoice jobs, then that would probably be handled in the processRecurringInvoices jobs.
3. you can use kue.Job.rangeByType() to search for currently active jobs. If a job is active, you can skip kicking it off again.