Question

I'm trying to create a file. In this file I put all the files that I need to process in a script and I pass the file as a parameter. My problem is that sometimes the list is not long enough to fill the buffer and nothing is written in disk. I tried to flush and fsync the tempfile but nothing happens. The Script is a third party script so I can't change the way the parameters are passed.

with tempfile.NamedTemporaryFile(bufsize=0) as list_file:
    list_file.write("\n".join(file_list) + "\n")
    list_file.flush()
    os.fsync(list_file)

    command = "python " + os.path.join(SCRIPT_PATH, SCRIPT_NAME) + " --thread --thread_file "
    ppss_command = [SCRIPT_PATH + "/ppss", "-f", list_file.name, "-c", command]
    p = subprocess.Popen(ppss_command)
    out,err = p.communicate()

Final Solution Code(jterrace Answer):

with tempfile.NamedTemporaryFile(delete=False) as list_file:
     list_file.write("\n".join(file_list) + "\n")
     list_file.close()
     command = "python " + os.path.join(SCRIPT_PATH, SCRIPT_NAME) + " --thread --thread_file "
     ppss_command = [SCRIPT_PATH + "/ppss", "-f", list_file.name, "-c", command]
     p = subprocess.Popen(ppss_command)
     out, err = p.communicate()
     list_file.delete = True
Was it helpful?

Solution

From the NamedTemporaryFile docstring:

Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).

So, while you still have the file open, it might not be able to be read from the subprocess. Here's what I'd do instead:

fd, tmp_fpath = tempfile.mkstemp()
os.close(fd) # Needed on Windows if you want to access the file in another process

try:
    with open(tmp_fpath, "wb") as f:
        f.write("\n".join(file_list) + "\n")

    # ... subprocess stuff ...
    do_stuff_with(tmp_fpath)
finally:
    os.remove(tmp_fpath)
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top