I've written some code to log on to a an AS/400 FTP site, move to a certain directory, and locate files I need to download. It works, but it seems like when there are MANY files to download I receive:
socket.error: [Errno 10054] An existing connection was
forcibly closed by the remote host
I log on and navigate to the appropriate directory successfully:
try:
newSession = ftplib.FTP(URL,username,password)
newSession.set_debuglevel(3)
newSession.cwd("SOME DIRECTORY")
except ftplib.all_errors, e:
print str(e).split(None,1)
sys.exit(0)
I grab a list of the files I need:
filesToDownload= filter(lambda x: "SOME_FILE_PREFIX" in x, newSession.nlst())
And here is where it is dying (specifically the newSession.retrbinary('RETR '+f,tempFileVar.write)
):
for f in filesToDownload:
newLocalFileName = f + ".edi"
newLocalFilePath = os.path.join(directory,newLocalFileName)
tempFileVar = open(newLocalFilePath,'wb')
newSession.retrbinary('RETR '+f,tempFileVar.write)
tempFileVar.close()
It downloads upwards of 85% of the files I need before I'm hit with the Errno 10054
and I guess I'm just confused as to why it seems to arbitrarily die when so close to completion. My honest guess right now is too many requests to the FTP when trying to pull these files.
Here's a screenshot of the error as it appears on my command prompt:
Any advice or pointers would be awesome. I'm still trying to troubleshoot this.