Emails being sent twice

Question

We have an email queue table in the database. It holds the subject, HTML body, to address, from address etc.

In Global.asax every interval, the Process() function is called which despatches a set number of emails. Here's the code:

namespace v2.Email.Queue
{
    public class Settings
    {
        // How often process() should be called in seconds
        public const int PROCESS_BATCH_EVERY_SECONDS = 1;

        // How many emails should be sent in each batch.  Consult SES send rates.
        public const int EMAILS_PER_BATCH = 20;
    }

    public class Functions
    {
        private static Object QueueLock = new Object();

        /// <summary>
        /// Process the queue
        /// </summary>
        public static void Process()
        {
            lock (QueueLock)
            {
                using (var db = new MainContext())
                {
                    var emails = db.v2EmailQueues.OrderBy(c => c.ID).Take(Settings.EMAILS_PER_BATCH);
                    foreach (var email in emails)
                    {
                        var sent = Amazon.Emailer.SendEmail(email.FromAddress, email.ToAddress, email.Subject,
                                                                email.HTML);
                        if (sent)
                            db.ExecuteCommand("DELETE FROM v2EmailQueue WHERE ID = " + email.ID);
                        else
                            db.ExecuteCommand("UPDATE v2EmailQueue Set FailCount = FailCount + 1 WHERE ID = " + email.ID);
                    }
                }
            }
        }

The problem is that every now and then it's sending one email twice.

Is there any reason from the code above that could explain this double sending?

Small test as per Matthews suggestion

const int testRecordID = 8296;
using (var db = new MainContext())
{
    context.Response.Write(db.tblLogs.SingleOrDefault(c => c.ID == testRecordID) == null ? "Not Found\n\n" : "Found\n\n");
    db.ExecuteCommand("DELETE FROM tblLogs WHERE ID = " + testRecordID);
    context.Response.Write(db.tblLogs.SingleOrDefault(c => c.ID == testRecordID) == null ? "Not Found\n\n" : "Found\n\n");
}
using (var db = new MainContext())
{
    context.Response.Write(db.tblLogs.SingleOrDefault(c => c.ID == testRecordID) == null ? "Not Found\n\n" : "Found\n\n");
}

Returns when there is a record:

Found

Found

Not Found

If I use this method to clear the context cache after the delete sql query it returns:

Found

Not Found

Not Found

However still not sure if it's the root cause of the problem though. I would of thought the locking would definitely stop double sends.

Answer 1

The issue that your having is due to the way Entity Framework does its internal cache.

In order to increase performance, Entity Framework will cache entities to avoid doing a database hit.

Entity Framework will update its cache when you are doing certain operations on DbSet.

Entity Framework does not understand that your "DELETE FROM ... WHERE ..." statement should invalidate the cache because EF is not an SQL engine (and does not know the meaning of the statement you wrote). Thus, to allow EF to do its job, you should use the DbSet methods that EF understands.

for (var email in db.v2EmailQueues.OrderBy(c => c.ID).Take(Settings.EMAILS_PER_BATCH))
{
    // whatever your amazon code was...

    if (sent)
    {
        db.v2EmailQueues.Remove(email);
    }
    else
    {
        email.FailCount++;
    }
}

// this will update the database, and its internal cache.
db.SaveChanges(); 

On a side note, you should leverage the ORM as much as possible, not only will it save time debugging, it makes your code easier to understand.