intermittent smtp broken pipe (errno 32)

Question

I have a python script that sends out a few sets of emails (to different addresses with different contents) using smtplib periodically throughout the day. Somewhat frequently (say about 1 in 5 times when sending out batches of more than one email at the same time), I get a IOError (errno: broken pipe). I try reseting and quitting the SMTP server and then connect to the server again and attempt to resend, but that always fails if it failed the first time (with the same exception). The SMTP server is maintained by the college and should be reliable (and allows loginless emails as long as you are on the intranet).

Ignoring the ugliness of the code below (lack of DRY), can anyone suggest a more reliable way to connect?

I create a class called EmailSet which will send a batch of a few emails that has a member function send_emails:

class EmailSet(object):
    ...
    def send_emails(self):
        try: # Connect to server
            server = smtplib.SMTP( smtp_server_name_str, 25)
            server.set_debuglevel(self.verbose)
            server.ehlo()
            for email in self.email_set:
                try: # send 1st mail
                    logging.debug("Sending email to %r" % email.recipients)        
                    response_dict = server.sendmail(email.fromaddr, email.recipients, email.msg_str())
                    logging.info("Sent email to %r" % email.recipients)        
                except Exception as inst:
                    logging.error('RD: %r' % response_dict)
                    logging.error("Email Sending Failed")
                    logging.error("%r %s" % ( type(inst), inst ) )
                    try: # send second mail
                        logging.info("Second Attempt to send to %r" % email.recipients)
                        try:
                            server.rset() 
                            server.quit()
                        except:
                            pass
                        time.sleep(60) # wait 60s
                        server = smtplib.SMTP( smtp_server_name_str, 25)
                        server.set_debuglevel(self.verbose)
                        server.ehlo()
                        response_dict = server.sendmail(email.fromaddr, email.recipients, email.msg_str())
                        logging.info("Sent email to %r (2nd Attempt)" % email.recipients)        
                    except Exception as inst:
                        try:
                            logging.error('RD: %r' % response_dict)
                        except:
                            pass
                        logging.error("Second Attempt Email Sending Failed")
        except:
            logging.debug("Can't connect to server")
        finally:
            logging.debug("Reseting and Quitting Server")
            server.rset()
            server.quit()
            logging.debug("Successfully Quit Server")
        return True

Any thoughts on how to proceed debugging this? The stmp server isn't maintained by me, though should be well-maintained (used for ~10k person organization). I originally connected and disconnected from the smtpserver after sending each email, but that produced more errors than this method.

Also would it be safer to use /usr/sbin/sendmail rather than smtplib?

Answer 1

Also would it be safer to use /usr/sbin/sendmail rather than smtplib?

From the point of view of handling a message queue and a queue-runner for retries. Yes, sendmail or another local MTA could handle this for you.

Any thoughts on how to proceed debugging this?

Packet capture. Use wireshark to capture the smtp traffic, see whats going on. You've got a lot of broad exception handling that isn't necessarily showing you the exact error.