Amazon Ses PHP SDK 2 - How to achieve high send rate

Question

Sending e-mails using the sendEmail method of the SesClient class from the PHP SDK currently takes about half a second per e-mail. I'm looping through the array of recipients, and setting the message ToAddresses property to equal an array containing just the recipient e-mail address before calling sendEmail(). I'd like to speed up the sending process but it seems to me that the PHP class provided by the SDK does one request per message (in my case, recipient). (Possibly one connection per message?)

I did my bit of reading, and I considered using the BccAddresses property to send e-mails in batches but we would like the To: header to be explictly set, instead of saying just "undisclosed-recipients", so I'd like to know if anyone has a better approach.

Answer 1

What I've done with PHP and AWS in the past is set up an email queue in the db and have multiple processes work off it to deliver emails.

Alternatively you can just deliver the emails in multiple threads (possibly non trivial in PHP). Here's an example in ruby with aws-sdk-ruby.

  require aws-sdk
  ses = AWS::SimpleEmailService.new(:access_key_id => 'ACCESS_KEY', :secret_access_key => 'SECRET_KEY')
  pool = Array.new(ses.quotas[:max_send_rate]) do |i|
    Thread.new do
      ses.send_email(
        :subject => 'subject',
        :body_html => 'body_html',
        :body_text => 'body_text',
        :to => 'mailto@example.com',
        :source => 'mailfrom@example.com',
      )
    end
  end

  sleep(1)
  pool.each(&:join)

Answer 2

For those that come stumbling through AWS and SES trying to solve this problem of parallel requests that are done on a persistent connection, AWS SDK 2 and up has support for this in php using command objects.

The SesClient and other clients can execute commands in parallel. This is the normal method for firing a single connection and email through SES:

$result = $client->sendEmail(array(
    //email data
));

The client objects are very powerful and inherit many methods to perform and manipulate the requests such as getCommand() and execute(). It took hours and hours of digging before I found the simple solution! You just have to know the right thing to search for. Here is an example:

$commands = array();
$commands[] = $sesClient->getCommand('SendEmail', array(
    //email data
));
$commands[] = $sesClient->getCommand('SendEmail', array(
    //email data
));

// Execute an array of command objects to do them in parallel
$sesClient->execute($commands);

// Loop over the commands, which have now all been executed
foreach ($commands as $command) {
    $result = $command->getResult();
    // Do something with result
}

Error handling can be achieved by executing the following code:

use Guzzle\Service\Exception\CommandTransferException;

try {
    $succeeded = $client->execute($commands);
} catch (CommandTransferException $e) {
    $succeeded = $e->getSuccessfulCommands();
    echo "Failed Commands:\n";
    foreach ($e->getFailedCommands() as $failedCommand) {
        echo $e->getExceptionForFailedCommand($failedCommand)->getMessage() . "\n";
    }
}

Amazon documents these examples in their developer guide under the commands feature.