which queue should I use (App Engine)

Question

I have app engine application. I have users refresh tokens (In order to have access to google drive) in my database.

Now, I want to create this:

Every week (I mean every 7th day), I want to temorary download users PDF documents from google drive and work with them. I send mails to each user about their pdf documents.

The main problem is that , there might be many users. and each user may have a lot of document too. I should do that work for each users, once a week. But each users data needs much time too.

QUESTION: So, now I think, which time service should I use? Cron or Task Queue. and why? and if Task Queue, which one. which will be faster, and flexible? I can send mail to the user later too (it's not necessary to send mails immediately when she/he requests)

QUESTION2: can I run Task Queue , for instance, once a week?

For example If I want to run it every day I can use something like that:

<rate>1/d</rate>

but how can I do that once a week?

QUESTION3: because of there might be many user (and because of each user needs much time), can I use something like that?

CRON job will be weekly (once a week). And CRON to call TASK QUEUSE, for each user. Each user data will be downloaded in the app engine server temporary (I think , If I save it in memory, it will be very hard for server). Then I will see PDF documents and send mails to each user. Is this good way? or should I use only CRON? did I have limitation here? On server storage or at queues or something like that.

Answer 1

Use both. Create a cron job to run every 7 days. Have the cron job fire a task (in the push queue) to process your PDFs. I'd use a separate task for each PDF to process, and configure your queue.yaml so that it processes them at the correct rate (depending on budget / rate limiting factors etc).

If you need to send mail, you can do this from the task request, via the mail api.

As a side note, if you have many users, a better approach may be to have the cron job run more frequently than every 7 days (say, once a day, or even more). You can use logic to determine which users need to be processed each time the cron runs. This may even the load, and ultimately save you money.