Subprocess seems to end, but does not release file handle

Question

Running under Windows 8, Python 3.3, I am spawning 3 processes that should run one after the other: first, signing an executable, second, building a kit using Inno Setup (it involves the executable from the 1st step), and finally, using the output from the 2nd step to sign it. However, from time ti time, I get what seems to be a race condition, in which the signing tool complains that it cannot sign the executable. It seems to me that the 2nd process does not release the file handle somehow (or the OS)...after looking up the error code, although it should be done (I am using process.communicate() to make sure of that). I also suspect a race condition since, using time.sleep() seems to solve the problem.

Is it possible that the 2nd process somehow detaches itself and runs in the background? But if it does, how come I still get to always see the operations logged in properly in my log file (e.g. 1st singning output, build, 2nd signing (regardless whether it is successful or unsuccesful))? Shouldn't the fact that I get output from communicate mean that all resources (including file handles) have been released? Here's the code:

def do_build():
'''
Prepare the kit.
'''
    global kit_file 
    kit_file = ''.join([OUTPUT_FILENAME, '_', version]) # do not add '.exe', as Inno does it during build
    DESCR = 'Execute Build'
    logf.write(BEGIN + DESCR + SEP2)

    def run_command(c, ex):
        with subprocess.Popen(c, stdout = subprocess.PIPE, stderr = subprocess.PIPE, executable = ex) as proc:  
            stdout_data, stderr_data = proc.communicate()
            logf.write(str(stdout_data, 'cp1252'))   
        if proc.returncode != 0:    # log errors if needed
            logf.write(str(stderr_data, 'cp1252'))
            sys.exit()       

    sign_exe = [SIGNCODE, '-cn', TTT, '-n', KIT_TYPE2.upper(), '-i', URL, '-t', TSURL]
    sign_kit = sign_exe[:]  # make copy
    sign_exe.append(os.sep.join([PDIR, DEPLOYMENT, EXECUTABLE]))  # sign the executable
    run_command(sign_exe, os.sep.join([PDIR, SIGNCODE]))

    compile = [ISCC, ''.join(['/O', OUTPUT_DIR]), ''.join(['/F', kit_file]), os.sep.join([PDIR, ISS_FILE])]   # compile using the ISS script
    run_command(compile, os.sep.join([ISSC_PATH, ISCC]))

    # time.sleep(something) here seems to save the day...

    sign_kit.append(os.sep.join([PDIR, ''.join([kit_file, '.exe'])]))   # sign the kit, don't forget '.exe'
    run_command(sign_kit, os.sep.join([PDIR, SIGNCODE]))
    logf.write(END + DESCR + SEP2)

I am using InnoSetup 5, the command-line tool, iscc.exe, FWIW.

Any explanation for this behavior? The only work-around I see is using os.access(file, os.W_OK) before attempting the last signing.

I am not running an antivirus and cannot think of anything else that could interfere with the signing process.

Answer 1

It turns out I was looking for the culprit in the wrong direction.

After more research - and failing to find out what process was locking my file (using Process Explorer, the Handle tool directly from my script, LockHunter) - I found enough anecdotal evidence to become certain that Windows Defender was the "offender". Apparently, it is so well integrated in Windows 8 that you can completely forget that its real-time protection module is running in the background. Sometimes, as soon as my file executable was created, it would lock it for an infinitesimal amount of time, just enough for my subsequent attempt to sign it...to fail.

I apologize for my initial, false presumption.