SMS and Email Queues from Database

Question 1

Thanks for the detailed response Mike. I finally ended up in implementing a REST API for my application with secure Username+Password+Key access and run it from a 3rd Party Service Iron.io which gets

www.example.com/rest/messages/format/json

It iterates and sends the messages collecting status in an array - which it then posts back to

www.example.com/rest/messagesposted

I followed this approach because I had to schedule messages for over 90 days interval and queues hold messages only for say like 14 days.

What do you recon?

Question 2

I had to do something similar and did as Charles Engelke suggested - I used SQS.

I eliminated the database entirely by putting the entire message contents in the SQS message. You're limited to 64k in an SQS message, so as long as thats not a problem this approach is possible.

Here is sample code to queue up the message:

package com.softwareconfidence.bsp.sending;

import com.amazonaws.services.sqs.AmazonSQS;
import com.amazonaws.services.sqs.model.SendMessageRequest;
import com.googlecode.funclate.json.Json;

import java.util.HashMap;
import java.util.Map;

public class EmailQueuer  {
    private final AmazonSQS sqs;
    private final String sendQueueUrl;

    public EmailQueuer(AmazonSQS sqs,String sendQueueUrl) {
        this.sqs = sqs;
        this.sendQueueUrl = sendQueueUrl;
    }

    public void queue() {
        Map<String,String> emailModel = new HashMap<String, String>(){{
            put("from","me@me.com");
            put("to","you@you.com");
            put("cc","her@them.com");
            put("subject","Greetings");
            put("body","Hello World");
        }}; 
        sqs.sendMessage(new SendMessageRequest(sendQueueUrl, Json.toJson(emailModel)));
    }
}

Then in your app you need to have an executor service that polls the queue and processes messages:

new ScheduledThreadPoolExecutor(1).scheduleAtFixedRate(sendEmails(), 0, 1, MINUTES)

You will need to make sure to call shutdown() on this executor when it app is exiting. Anyway, this line will send emails every minute, where sendEmails() returns an instance of this Runnable class:

package com.softwareconfidence.bsp.standalone.sending;

import com.amazonaws.services.simpleemail.AmazonSimpleEmailService;
import com.amazonaws.services.simpleemail.model.*;
import com.amazonaws.services.sqs.AmazonSQS;
import com.amazonaws.services.sqs.model.DeleteMessageRequest;
import com.amazonaws.services.sqs.model.ReceiveMessageRequest;
import com.amazonaws.services.sqs.model.ReceiveMessageResult;
import com.amazonaws.services.sqs.model.SendMessageRequest;
import com.googlecode.funclate.json.Json;

import java.io.PrintWriter;
import java.io.StringWriter;
import java.util.List;
import java.util.Map;

public class FromSqsEmailer implements Runnable {
    private final AmazonSQS sqs;
    private final String sendQueueUrl;
    private final String deadLetterQueueUrl;
    private final AmazonSimpleEmailService emailService;

    public FromSqsEmailer(AmazonSimpleEmailService emailService, String deadLetterQueueUrl, String sendQueueUrl, AmazonSQS sqs) {
        this.emailService = emailService;
        this.deadLetterQueueUrl = deadLetterQueueUrl;
        this.sendQueueUrl = sendQueueUrl;
        this.sqs = sqs;
    }

    public void run() {
        int batchSize = 10;
        int numberHandled;
        do {
            ReceiveMessageResult receiveMessageResult =
                    sqs.receiveMessage(new ReceiveMessageRequest(sendQueueUrl).withMaxNumberOfMessages(batchSize));
            final List<com.amazonaws.services.sqs.model.Message> toSend = receiveMessageResult.getMessages();
            for (com.amazonaws.services.sqs.model.Message message : toSend) {
                SendEmailResult sendResult = sendMyEmail(Json.parse(message.getBody()));
                if(sendResult != null) {
                    sqs.deleteMessage(new DeleteMessageRequest(sendQueueUrl, message.getReceiptHandle()));
                }
            }
            numberHandled = toSend.size();
        } while (numberHandled > 0);
    }

    private SendEmailResult sendMyEmail(Map<String, Object> emailModel) {
        Destination to = new Destination()
                .withToAddresses(get("to", emailModel))
                .withCcAddresses(get("cc", emailModel));
        try {
            return emailService.sendEmail(new SendEmailRequest(get("from", emailModel), to, body(emailModel)));
        } catch (Exception e){
            StringWriter stackTrace = new StringWriter();
            e.printStackTrace(new PrintWriter(stackTrace));
            sqs.sendMessage(new SendMessageRequest(deadLetterQueueUrl, "while sending email " + stackTrace));
        }
        return null;
    }

    private String get(String propertyName, Map<String, Object> emailModel) {
        return emailModel.get(propertyName).toString();
    }

    private Message body(Map<String, Object> emailModel) {
        Message message = new Message().withSubject(new Content(get("subject", emailModel)));
        Body body = new Body().withText(new Content(get("body", emailModel)));
        message.setBody(body);
        return message;
    }
}

One downsize of this approach if you're using a database is that the email sending step is a HTTP call. If you have a database transaction that rollsback after this HTTP call, your business process is undone, but the email is going to be sent.

Food for thought.