Script to move messages from one IMAP server to another

Question

Our office uses 2 IMAP servers for e-mail, one is the incoming server and holds the recent e-mails and the other is an archive server. We mainly use Outlook 2010 and our current process is to periodically drag sent messages from the incoming server to the archive.

Today I was asked into looking into writing a script and that would periodically (probably using crontab) grab all sent messages and move them to archive.

I've looked into some example of SSL or telnet to access the server and poke around. However I don't know the best way to script this or how to move files cross server within the IMAP environment.

What's the best way to accomplish this? I'd prefer to use Python just from comfort level, but if there is already an existing solution in another language, I could deal with it.

---

Update:

Ok, here's some code. Currently It copies the messages just fine, however, it will duplicate exisiting messages on the archive server.

import imaplib
import sys

#copy from
f_server = 'some.secret.ip.address'
f_username = 'j@example.com'
f_password = 'password'
f_box_name = 'Sent Messages'

#copy to
t_server = 'archive.server.i.p'
t_username = 'username'
t_password = 'password'
t_box_name = 'test'

To = imaplib.IMAP4(t_server) 
To.login(t_username, t_password)
print 'Logged into mail server'

From = imaplib.IMAP4(f_server)
From.login(f_username, f_password)
print 'Logged into archive'

From.select(f_box_name)  #open box which will have its contents copied
print 'Fetching messages...'
typ, data = From.search(None, 'ALL')  #get all messages in the box
msgs = data[0].split()

sys.stdout.write(" ".join(['Copying', str(len(msgs)), 'messages']))

for num in msgs: #iterate over each messages id number
    typ, data = From.fetch(num, '(RFC822)')
    sys.stdout.write('.')
    To.append(t_box_name, None, None, data[0][1]) #add a copy of the message to the archive box specified above

sys.stdout.write('\n')

try:
    From.close()
From.logout()

try:
    To.close()
To.logout()

Some sources:
Doug Hellman's Blog: imaplib - IMAP4 Client Library
Tyler Lesmann's Blog: Copying IMAP Mailboxes with Python and imaplib

I still need to:

• delete/expunge messages on the live server
• not copy duplicates (actually this would be fixed by deleting originals after copying, but...)
• error trapping

---

Update 2:

Anyone have any ideas on how to not create duplicates when copying? (excluding the option of deleting originals, for now) I thought about searching text, but realized nested replies could throw that off.

Answer 1

Here's what I ended up using. I don't claim that it's perfect, the way it uses msg_num and not id is a little risky. But this is fairly low volume moves, maybe a couple an hour (on cron).

import imaplib

#copy from
from_server = {'server': '1.1.1.1',
               'username': 'j@example.com',
               'password': 'pass',
               'box_names': ['Sent', 'Sent Messages']}

#copy to
to_server = {'server': '2.2.2.2',
             'username': 'archive',
             'password': 'password',
             'box_name': 'Sent'}

def connect_server(server):
    conn = imaplib.IMAP4(server['server']) 
    conn.login(server['username'], server['password'])
    print 'Logged into mail server @ %s' % server['server']
    return conn

def disconnect_server(server_conn):
    out = server_conn.logout()

if __name__ == '__main__':
    From = connect_server(from_server)
    To = connect_server(to_server)

    for box in from_server['box_names']:
        box_select = From.select(box, readonly = False)  #open box which will have its contents copied
        print 'Fetching messages from \'%s\'...' % box
        resp, items = From.search(None, 'ALL')  #get all messages in the box
        msg_nums = items[0].split()
        print '%s messages to archive' % len(msg_nums)

        for msg_num in msg_nums:
            resp, data = From.fetch(msg_num, "(FLAGS INTERNALDATE BODY.PEEK[])") # get email
            message = data[0][1] 
            flags = imaplib.ParseFlags(data[0][0]) # get flags
            flag_str = " ".join(flags)
            date = imaplib.Time2Internaldate(imaplib.Internaldate2tuple(data[0][0])) #get date
            copy_result = To.append(to_server['box_name'], flag_str, date, message) # copy to archive

            if copy_result[0] == 'OK': 
                del_msg = From.store(msg_num, '+FLAGS', '\\Deleted') # mark for deletion

        ex = From.expunge() # delete marked
        print 'expunge status: %s' % ex[0]
        if not ex[1][0]: # result can be ['OK', [None]] if no messages need to be deleted
            print 'expunge count: 0'
        else:
            print 'expunge count: %s' % len(ex[1])

    disconnect_server(From)
    disconnect_server(To)

Answer 2

I'm not sure what volume of messages you're dealing with, but you could extract the Message-ID from each one and use that to find out if it's a duplicate. Either generate a list of IDs already on the target server each time you prepare to move messages, or add them to a simple database as they are archived.

You could narrow things down by an additional message property like Date if the lookups are too expensive, then drop the older lists when you no longer need them.

Answer 3

Presumably too late to be helpful to the OP, but hopefully useful for anyone following along after now.

This looks like a generic requirement. You probably shouldn't be custom coding anything.

You would probably be better off using an MTA configured to send copies of everything to an archive as well as sending stuff to your IMAP server. If this is hard for you to set up, consider using a third party service, who would manage your archives, and forward mail on to your existing mail server.

If you really do want to do this by copying from IMAP, I'd suggest looking at offlineimap.

If you really do want to do it yourself, the way to track the messages you've already seen is by using the Message-ID header.