How to efficiently verify existence of large volume of files using JCIFS library?

Question

I have a process that reads database to gather the server and path information of about 200,000 files (and growing). I used JCIFS library to check if the file exists on the designated location one at a time using something like:

SmbFile file = new SmbFile(fullPath, getNtlmPasswordAuthentication());
if(file.exists()) {
     return true;
}

It takes couple hours to complete the process. I'm trying to find a way to speed up the process. The files needed to be verified spread over 40 directories. Each directory can contain couple thousand of files. The SmbFile API has a listFiles() function that allows me to open a directory and get back an array of SmbFile in that folder. I wonder if I'm on the right track and if anyone has better idea. Thanks!

Answer 1

Absolutely you should use listFiles(). If I'm understanding things correctly, your approach results in a request-per-file, and listFiles() will give you a request-per-directory - I'd expect speed up of x1000 or so