memory leak with symfony2 & swiftmailer

Question

I'm using a symfony2 command as a cron job to send bulk email to members of a site.

Actual Code:

$output->writeln('Before: '.(memory_get_usage() / 1024 / 1024));

$mailer->send($m);

$output->writeln('After: '.(memory_get_usage() / 1024 / 1024));

My results:

Before: 182.38 MB
After: 183.73 MB

Every time I send an email, swiftmailer is consuming an additional 1+MB of memory. That just doesn't seem right, but the memory increases every single time a new message is sent. Am I doing something wrong here?

Answer 1

Edit: Not sure why users are downvoting. The fact is that swiftmailer itself has a memory leak directly related to its events/plugin system. When hooks are executed it eats away memory which never gets released. Solely removing the events system alone fixed the memory leak without breaking the way swiftmailer works when it comes to actually sending email.

I fixed the problem. I edited the MailTransport class and removed the events system in swiftmailer. I am only allowing the message itself to be built and sent, none of the before and after plugin crap.

I took the pieces from the throttlerplugin and created by own version of it which specifically will only handle messages per minute:

class EmailThrottler {

    private $startTime;
    private $messagesPerMinute;
    private $_messages;

    public function __construct($messagesPerMinute = 25)
    {
        $this->startTime = time();
        $this->messagesPerMinute = $messagesPerMinute;
        $this->_messages = 0;
    }

    public function run()
    {
        $this->_messages++;
        $duration = time() - $this->startTime;
        $sleep = $this->_throttleMessagesPerMinute($duration);

        if ($sleep > 0) {
            sleep($sleep);
        }
    }

    private function _throttleMessagesPerMinute($duration)
    {
        $expectedDuration = $this->_messages / ($this->messagesPerMinute / 60);
        return (int) ceil($expectedDuration - $duration);
    }

}

I initialized my custom throttler class before the mailing loop:

$throttler = new EmailThrottler($pendingCampaign->getRateLimit());

and I run it after every sent email:

$mailer->send($m);
$throttler->run();

Hopefully they find a fix to their events system. I am going to see if the performance is any different on 5.4 anyway, but to those who are on 5.3 this solution I just detailed worked for me on 5.3

Cheers :)

Answer 2

SwiftMailer memory spooling system

The problem you're pointing is actually more a solution than a problem. You're not doing anything wrong, and it's due to SwiftMailer's internal way of spooling sent emails.

Upon send() method, SwiftMailer doesn't actually send anything, instead simply place it into his spool. By default, the spooling option is memory, and spool flush happens right before the Kernel terminates. So there is no chance that memory_get_usage() will tell you that memory has been released (because obviously, your script is still running and SwiftMailer hasn't flushed its spool yet).

From Symfony2 documentation:

When you use spooling to store the emails to memory, they will get sent right before the kernel terminates. This means the email only gets sent if the whole request got executed without any unhandled Exception or any errors.

Using file spooling system

If this spooling option causes you trouble, you can switch to the File spooling option.

It can be done by changing this:

# app/config/config.yml
swiftmailer:
    # ...
    spool: { type: memory }

To this:

# app/config/config.yml
swiftmailer:
    # ...
    spool:
        type: file
        path: /path/to/spool

Then, you can configure a cron job to automatically and periodically flush the spool:

$ php app/console swiftmailer:spool:send --env=prod

Using no spool

Or, you can choose to use no spool system at all, which means that all you email will be sent during the execution of your script. It will get you rid of memory problems, but may potentially hurt your users depending on if you're sending mail while they're requesting a page. But as you're doing this by cron job, in your case it certainly won't be a problem. Could be one day, though.

Answer 3

I got a similar memory problem sending emails with swiftmailer. It seems that the plugins create these memory leaks, so is not necessary to remove the event dispatcher code, simply disable the plugins (and as for your case, patch with your own solution).

Using simfony2, disable logging is easy, in the config:

swiftmailer:
    logging: false

or with multiple mailers:

swiftmailer:
    mailers:
        my_mailer:
            logging: false

In my case, sending ~1500 emails, the memory was growing up to ~100MB (just because of the logger), now >0.2MB .

I know it's a little bit late, but maybe this can be useful to someone that get into the same problem. :-)