Moving files and creating directories if certain file type in python

Question

This is probably a simple question, but I'm brand new to python and programming in general.

I'm working on a simple program to copy/move .mp3 files from on location to another while mirroring the directory structure of the source location. What I have so far works, however it also creates new folders in the destination location even if the source folder contained no mp3 files. I only want to create the new directories if the source contains .mp3s, otherwise it could lead to a bunch of empty folders in the destination.

Here is what I have so far:

import os
import shutil #Used for copying files

##CONFIG
source_dir = "C:\Users\username\Desktop\iTunes\\" #set the root folder that you want to     scan and move files from.  This script will scan recursively.
destPath = "C:\Users\username\Desktop\converted From iTunes" #set the destination root that you want to move files to.  Any non-existing sub directories will be created.
ext = ".mp3" #set the type of file you want to search for.
count = 0 #initialize counter variable to count number of files moved
##

##FIND FILES
for dirName, subdirList, fileList in os.walk(source_dir):

    #set the path for the destination folder(s)
    dest = destPath + dirName.replace(source_dir, '\\') 

    #if the source directory doesn't exist in the destination folder
    #then create a new folder
    if not os.path.isdir(dest):
        os.mkdir(dest)
        print('Directory created at: ' + dest)

    for fname in fileList:
        if fname.endswith(ext) :
            #determine source & new file locations
            oldLoc = dirName + '\\' + fname
            newLoc = dest + '\\' + fname

            if os.path.isfile(newLoc): # check to see if the file already exists.  If it does print out a message saying so.
                print ('file "' + newLoc + fname + '" already exists')

            if not os.path.isfile(newLoc): #if the file doesnt exist then copy it and print out confirmation that is was copied/moved
                try:
                    shutil.move(oldLoc, newLoc)
                    print('File ' + fname + ' copied.')
                    count = count + 1
                except IOError:
                    print('There was an error copying the file:  "' + fname + '"')
                    print 'error'            

print "\n"
print str(count) + " files were moved."
print "\n"

so if the folder structure is something like:

root->
 band 1->
  album name->
   song.m4a,
   song2.m4a

right now it will create all those folders in the destination driectory, even though there are no .mp3s to copy.....

Any help is appreciated!

Answer 1

The simplest thing to do I can think of for your existing code would be to just make it skip over any folders that don't have any .mp3 files in them. This can easily be done by adding the following items and if statement to the top of your loop. The itertools.ifilter() and fnmatch.fnmatch() functions can be used together to simplify checking for files with the proper extension.

from itertools import ifilter
from fnmatch import fnmatch

ext = '.mp3'
fnPattern = '*'+ext

for dirName, subdirList, fileList in os.walk(source_dir):
    if not any(ifilter(lambda fname: fnmatch(fname, fnPattern), fileList)):
        print '  skipping "{}"'.format(dirName)
        continue
    ...

You will also have to change the os.mkdir(dest) to os.makedirs(dest) in the code further down to ensure that any subdirectories skipped by earlier iterations get created when there's a need to copy files to a corresponding subbranch of the destination directory.

You could optimize things a bit by creating and saving a possibly empty iterator of matching files that have the extension, and then use it again later to to determine what files to copy:

from itertools import ifilter
from fnmatch import fnmatch

ext = '.mp3'
fnPattern = '*'+ext

for dirName, subdirList, fileList in os.walk(source_dir):

    # generate list of files in directory with desired extension
    matches = ifilter(lambda fname: fnmatch(fname, fnPattern), fileList)

    # skip subdirectory if it does not contain any files of interest
    if not matches:
        continue
    ...
    ... create destination directory with os.makedirs()
    ...
    #  copy each file to destination directory
    for fname in matches:
      ... copy file

Answer 2

I think I would create my own wrapper around copy for this sort of thing:

def fcopy(src,dest):
    """
    Copy file from source to dest.  dest can include an absolute or relative path
    If the path doesn't exist, it gets created
    """
    dest_dir = os.path.dirname(dest)
    try:
        os.makedirs(dest_dir)
    except os.error as e:
        pass #Assume it exists.  This could fail if you don't have permissions, etc...
    shutil.copy(src,dest)

Now you can just walk the tree calling this function on any .mp3 file.

Answer 3

Would shutils.copytree not do what you want in fewer lines?