How to implement a mail queue that can be easily integrated with SendGrid

Question

The goal: Implementing a modular, re-usable, object-oriented class, or package, that will be in charge of automating the process of user notifications. Those notifications are send according to a certain set of rules, to multiple users, in the background as I've already mentioned.

For instance: each time a new user is signing up, and there is a system message defined, and there are more than 10 active users per that super-user, then this system message is sent to all of the users.

Those 'rules' will probably be implemented as method, or sub - classes with an SQL model inside them.

Environment: LAMP, PHP 5.3. The preferred implementation would be in PHP. SendGrid mail delivery service + their official PHP library.

Implementation(what I thought about so far): I thought about using a cronjob or an Apache HTTP worker, the thing is that this model will be responsible for all of the email notifications of the system, and in that case it will have to run each 60 seconds or so, in order to ensure delivery on time and synchronically. To add or remove the cronjobs, I thought about using PHP's exec() or system(), although I am sure that there is a better way, perhaps editing the cron jobs file directly with the file functions?.

Issues: Any implementation should be able to integrate with SendGrid, since this is the delivery system that we are currently using, and it's proven to be efficient.

This is the heart of the system, so it needs to be as fail - proof as possible, and have the ability to self-diagnostic, otherwise nothing good will come out of it, and it would be better to send those email notifications directly through the pages which the users visit, and compromise on loading times, design etc.

Any ideas?, am I on the right direction at all?. I was looking for something that was already implemented on github and the rest of the web, but there is nothing that matches my needs and requirements. SendGrid also don't seem to offer a built - in queue.

Any help is much appreciated!.

Answer 1

I work at SendGrid.

The most robust solution would be to not implement the queue in code but rather to setup a local mail server and let that manage your email queueing. That mail server would then be configured to use SendGrid as a smart relay that provides all the other features.

This has several benefits:

• Very low latency between your application your local mail server

• Your local mail server can manage the queueing.

• SMTP Command pipelining reduces the number of protocol requests

  We have configuration guides for mail servers that can point you in the right direction.

In terms of the logic that triggers the sends to this local server, I think you're on the right track. You could build a background process system with cron, or you could use something like php-resque that can abstract some things and get you more control from within your code. php-resque or a similar solution might give you enough control that you feel confident not using a local MTA.