Python FTP "ERRNO 10054" Sequential File Download

Question

I've written some code to log on to a an AS/400 FTP site, move to a certain directory, and locate files I need to download. It works, but it seems like when there are MANY files to download I receive:

socket.error: [Errno 10054] An existing connection was 
              forcibly closed by the remote host

I log on and navigate to the appropriate directory successfully:

try:
    newSession = ftplib.FTP(URL,username,password)
    newSession.set_debuglevel(3)
    newSession.cwd("SOME DIRECTORY")
except ftplib.all_errors, e:
    print str(e).split(None,1)
    sys.exit(0) 

I grab a list of the files I need:

filesToDownload= filter(lambda x: "SOME_FILE_PREFIX" in x, newSession.nlst())

And here is where it is dying (specifically the newSession.retrbinary('RETR '+f,tempFileVar.write)):

for f in filesToDownload:
    newLocalFileName = f + ".edi"
newLocalFilePath = os.path.join(directory,newLocalFileName)
tempFileVar = open(newLocalFilePath,'wb')
newSession.retrbinary('RETR '+f,tempFileVar.write)
tempFileVar.close()

It downloads upwards of 85% of the files I need before I'm hit with the Errno 10054 and I guess I'm just confused as to why it seems to arbitrarily die when so close to completion. My honest guess right now is too many requests to the FTP when trying to pull these files.

Here's a screenshot of the error as it appears on my command prompt:

Any advice or pointers would be awesome. I'm still trying to troubleshoot this.

Answer 1

There's no real answer to this I suppose, it seems like the client's FTP is at fault here it's incredibly unstable. Best I can do is a hacky work around catching the thrown socket error and resuming where I left off in the previous session before being forcibly disconnected. Client's IT team is looking into the problem on their end finally.

Sigh.