Wait for external program to finish writing to file

Question

I have a Python script that calls an external program (sox to be precise). Now I have to do several things with sox, but always have to wait until one file is done writing so I can use it as an input file in my next command.

subprocess.wait() doesn't work, because the execution of sox will be done, but the file won't be done writing.

Here is the code I have:

import tempfile
import shlex

file_url = '/some/file.wav'
out_file = 'some/out.wav'
tmp_file = tempfile.NamedTemporaryFile()
pad_cmd = 'sox ' + file_url + ' ' + tmp_file.name + ' pad 0.0 3.0'
subprocess.call(shlex.split(pad_cmd))
trim_cmd = 'sox ' + tmp_file.name + ' -t wav ' + out_file + ' trim 0.0 3.0'

Any way to wait for the file to finish writing to tmp_file.name without using a timer that waits for a fixed amount of time. Or is there a built-in way in sox to use the output file as input? The special filename - didn't work for pad and trim.

Answer 1

If sox works with stdin/stdout:

#!/bin/sh
sox input.wav -t wav - pad 0.0 3.0 | sox -t wav - out.wav trim 0.0 3.0

then you could write it in Python:

#!/usr/bin/env python
from subprocess import Popen, PIPE

pad = Popen(["sox", "input.wav", "-t", "wav", "-", "pad", "0.0", "3.0"],
            stdout=PIPE)
trim = Popen(['sox', '-t', 'wav', '-', 'out.wav', 'trim', '0.0', '3.0'],
             stdin=pad.stdout)
pad.stdout.close() # notify pad on write if trim dies prematurely
pipestatus = [p.wait() for p in [pad, trim]]

Note: if you don't use an untrusted input to construct the commands then you could pass them as a single line for readability (the shell creates the pipe in this case):

#!/usr/bin/env python
from subprocess import check_call

cmd = "sox input.wav - -t wav pad 0.0 3.0 | sox - -t wav out.wav trim 0.0 3.0"
check_call(cmd, shell=True)

---

If you can't make it to write/read to/from stdout/stdin then you could try a named pipe, to avoid waiting for a file:

#!/bin/sh
mkfifo /tmp/sox.fifo
sox /tmp/sox.fifo -t wav out.wav trim 0.0 3.0 & # read from fifo
sox input.wav /tmp/sox.fifo pad 0.0 3.0         # write to fifo

or the same in Python:

#!/usr/bin/env python
from subprocess import Popen, check_call

with named_pipe() as path:
    trim = Popen(["sox", path, '-t', 'wav', 'out.wav', 'trim', '0.0', '3.0'])
    check_call(["sox", "input.wav", path, "pad", "0.0", "3.0"]) # pad
rc = trim.wait()

where named_pipe() is defined as:

import os
from contextlib import contextmanager
from shutil     import rmtree
from tempfile   import mkdtemp

@contextmanager
def named_pipe():
    dirname = mkdtemp()
    try:
        path = os.path.join(dirname, 'named_pipe')
        os.mkfifo(path)
        yield path
    finally:
        rmtree(dirname)

Answer 2

You can wait file before it will be avaiable for reading:

handle = open(tmp_file.name)
from subprocess import select
select.select([handle], [], [])