Errno 22 with Python shutil and .move() function

Question

Alright, so I've been debugging this for most of the afternoon and I'm throwing in the towel.

I'm trying to create a system that will help automate putting work folders back on the server after my student workers are done with them. I've gotten everything down so far--it correctly identifies the matching server folder, and it is able to move the server folder into an Archive folder to temporarily back it up while I copy the new work back.

However, it's that last bit that's holding me up. Everything is working beautifully, until I get to the part where I copy the folder that's passed as an argument back onto the server.

Here is the problem piece of code:

print "Match Source: " + match_src

archive_folder_name = match_src.rsplit('/', 1)[1]

print "Archive Folder Name: " + archive_folder_name

if not (os.path.isdir(os.path.join(os.path.dirname(match_src), "Archive"))):
    os.mkdir(os.path.join(os.path.dirname(match_src), "Archive"))

archive_dst = os.path.join(os.path.join(os.path.dirname(match_src), "Archive"), archive_folder_name)

print "Archived Folder Destination: " + archive_dst

# okay, archive folder has been made. Now we need to move the old
# stuff to this folder.

shutil.move(match_src, archive_dst)

# okay, archive folder is filled. Now to move the new stuff there
print "Updated Source: " + updated_folder_path

print "Destination: " + os.path.dirname(match_src)

shutil.move(updated_folder_path, os.path.dirname(match_src))

And here is the output from those print statements, and the error code:

ServerReturn mwl36$ python serverreturn_main.py /Users/mwl36/Desktop/Week\ 1 
I'm on a Mac.
Path: /Users/mwl36/Desktop/Week 1
Folder: Week 1
Match Source: /Volumes/GCPSX Mac HD/ID/Marisa/Work for Student Workers/201315/SERVERTEST/Week 1
Archive Folder Name: Week 1
Archived Folder Destination: /Volumes/GCPSX Mac HD/ID/Marisa/Work for Student Workers/201315/SERVERTEST/Archive/Week 1
Updated Source: /Users/mwl36/Desktop/Week 1
Destination: /Volumes/GCPSX Mac HD/ID/Marisa/Work for Student Workers/201315/SERVERTEST
Traceback (most recent call last):
  File "serverreturn_main.py", line 124, in <module>
    main()
  File "serverreturn_main.py", line 117, in main
    shutil.move(updated_folder_path, os.path.dirname(match_src))
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 296, in move
    copytree(src, real_dst, symlinks=True)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 206, in copytree
    raise Error, errors
shutil.Error: [('/Users/mwl36/Desktop/Week 1/testfile.txt', '/Volumes/GCPSX Mac HD/ID/Marisa/Work for Student Workers/201315/SERVERTEST/Week 1/testfile.txt', "[Errno 22] Invalid argument: '/Volumes/GCPSX Mac HD/ID/Marisa/Work for Student Workers/201315/SERVERTEST/Week 1/testfile.txt'")]

Where am I going wrong? If the call to do shutil.move(src, dst) worked the first time without issues, why is it barfing here? I've tried adding in a delay after the initial call, to no effect.

EDIT: What's even stranger is that it will actually perform the move, but never seems to delete the old folder on the desktop. Essentially, it copies the folder to the DST location, but doesn't clean up after itself.

EDIT #2: Calling os.listdir shows that the directory is gone, and that the only thing in there prior to the second move call is 'Archive'. I will test out copy() tomorrow when I'm at work and have access to the code again.

Answer 1

shutil.move() will copy the source to destination and then remove the source if "a simple move" is not possible. This can happen when the source and the destination are not on the same filesystem for example.

In this case, shutil.move() is doing just that: first it copies the source to the destination, and then does a rmtree() on the source.

During the copy, shutil will copy the bytes of the source files and then copy "stat" information (permissions, modification times, etc.). So, it seems like [Errno 22] Invalid argument is either coming from copying the bytes, or copying the stat data. It seems very unlikely that copying bytes would cause EINVAL error, so the most likely explanation is that copying of "stat" data is causing this error.

You can test this by adding something like this at the beginning of your main script:

import shutil
import sys
import traceback

orig_copystat = shutil.copystat
def my_copystat(src, dst):
    try:
        orig_copystat(src, dst)
    except Exception as e:
        sys.stdout.write('copystat failed: %s\n' % e)
        traceback.print_exc()

shutil.copystat = my_copystat

# Your script here...

If my guess is correct, you will see copystat failed: ... message(s) (and stack trace) when you run your script, but it should "work" otherwise. If you see the error message from the above function, you can change the above function to silently ignore the exception and run your script. Otherwise, try monkey-patching shutil.copyfile as above and see where exactly in copyfile() is the exception being raised.