Effective way to find total number of files in a directory

Question

I am building a Music file organizer(in python2) in which I read the metadata of all files & then put those file in the required folder.

Now, I am already ready with the command line interface but this script shows feedback in a way that it shows "Which file is it working on right now?".

If the directory contains say 5000 mp3 files, there should be some kind of feedback.

So, I would like to know the most efficient way to find the total number of mp3s available in a directory (scanning recursively in all subsequent directories too).

My idea is to keep track of the total files processed and show a progress bar according to that. Is there a better way (performance wise), please feel free to guide.

I want my app to not have any kind of platform dependent code. If there is serious performance penalty sticking to this idea, please suggest for linux.

Answer 1

I'm sorry to say this but no there isn't any way to do it more efficiently than recursively finding the files (at least that is platform (or filesystem) independent).

If the filesystem can help you it will, and you can't do anything to help it.

The reason it's not possible to do it without recursive scanning is how the filesystem is designed. A directory can be seen as a file, and it contains a list of all files it contains. To find something in a subdirectory you have to first open the directory, then open the subdirectory and search that.

Answer 2

@shadyabhi: if you have many subdirectories maybe you can speedup the process by using os.listdir and multiprocessing.Process to recurse into each folder.