I assume you want to run 18 processes (one process per file) with no more than N_CORES
processes in parallel.
The simplest way could be to use multiprocessing.Pool
here:
import multiprocessing as mp
import subprocess
def process_file(filename):
try:
return filename, subprocess.call([cmd, filename], cwd=localdir)
except OSError:
return filename, None # failed to start subprocess
if __name__ == "__main__":
pool = mp.Pool()
for result in pool.imap_unordered(process_file, filenames):
# report result here