Python Process throw a memory allocation trying to write a file, but there is plenty of memory available--What's the cause?

Question

I have a Python process that reads files from a database, and then writes them out to a network volume via fout:

eg:

fout = open((strOutputPath + strOutputName), 'wb')
for row in tupSQLResults :
    fout.write(row[0])

fout.close()

99% of the time, this is successful. However, on some occasions, I get a lovely: [Errno 12] Cannot allocate memory via str(sys.exc_info()[1])

When this occurs, I've checked the machine's memory, and usually the Python process is only using something like 36mb(similar virtual memory), with 3+gbs of free RAM. It will continue to the next files, and generally output them fine. (It's a scheduled task that runs every 5 mins, and outputs any files that have a newer timestamp in the db.)

Any thought as to what would then be causing this memory allocation error? Could it be a problem with the destination volume? If so, is there a more reliable transport mechanism (should I fout locally, and then copy using scp or similar?)

Thanks!

Answer 1

It may be that the network volume is unavailable for some reason.

I would catch the exception, sleep the process for a little while, and try it again.