Passing parameters to Python subprocess.Popen

Question

I'm converting this bash script to Python. I have a working Python version now. However, in order to get it to work, I had to hack the command I passed to subprocess.Popen() by making it into one long string. I do not wish to use one long command string. I wish to break this into the proper individual parameters. How can I do that in this specific example?

My specific question is how do I change this line:

process = subprocess.Popen(cmd, shell=True, ...

into a form like this:

process = subprocess.Popen([prog, inf, arate, outf], shell=False, ...

Here's my complete code so that the above fragments will make sense:

#!/usr/bin/env python

import sys
import subprocess
import os.path
import argparse #requires Python 2.7 or above

parser = argparse.ArgumentParser(description='Converts (.M4V) videos into (.MP3) audio files using ffmpeg.')
parser.add_argument('-d','--directory', help='Source directory name',required=True)
parser.add_argument('-o','--output',help='Output directory name', required=False)
parser.add_argument('-a','--arate',help='ffmpeg\'s arate option', default="64K", required=False)
parser.add_argument('-s','--source',help='Input type; e.g., file ext (default .m4v)', default=".m4v", required=False)
parser.add_argument('-e','--ext',help='Output type; e.g., file ext (default .mp3)', default=".mp3", required=False)
parser.add_argument('-p','--program',help='application program e.g., /usr/bin/ffmpeg', default="/usr/bin/ffmpeg", required=False)
args = parser.parse_args()

arate = args.arate
source = args.source
new_ext = args.ext                                                                                                                                             
prog = args.program
in_path = args.directory

if(not args.output):
    out_path = in_path
else:
    out_path = args.output

if(not os.path.isfile(prog)):
    print("Command {} does not exist".format(prog))

else:
    try:
        filenames = os.listdir(in_path)
        for file in filenames:
            print("name={}\nextension={}".format(name, extension))
            if(extension == source):
                inf = os.path.join(in_path, file)
                outf = os.path.join(out_path, name + new_ext)
                cmd = "{xprog} -i '{xpath}' -b:a {art} -vn '{xout}'".format(xprog=prog, xpath=inf, art=arate, xout=outf)
                process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
                stdout,stderr = process.communicate()
                return_code = process.poll()
                print("stdout='{}'\nstderr='{}'\nreturn_code='{}'".format(stdout, stderr, return_code))
    except IOError as e:
        print("Error: file not found.")

Answer 1

Here is a complete working solution with proper arguments for Popen().

import subprocess
import os.path
import argparse #requires Python 2.7 or above

parser = argparse.ArgumentParser(description='Converts (.M4V) videos into (.MP3) audio files using ffmpeg.')
parser.add_argument('-d','--directory', help='Source directory name',required=True)
parser.add_argument('-o','--output',help='Output directory name', required=False)
parser.add_argument('-a','--arate',help='ffmpeg\'s arate option', default="64K", required=False)
parser.add_argument('-s','--source',help='Input type; e.g., file ext (default .m4v)', default=".m4v", required=False)
parser.add_argument('-e','--ext',help='Output type; e.g., file ext (default .mp3)', default=".mp3", required=False)
parser.add_argument('-p','--program',help='application program e.g., /usr/bin/ffmpeg', default="/usr/bin/ffmpeg", required=False)
args = parser.parse_args()

arate = args.arate
source = args.source
new_ext = args.ext                                                                                                                                             
prog = args.program
in_path = args.directory

if(not args.output):
    out_path = in_path
else:
    out_path = args.output

if(not os.path.isfile(prog)):
    print("Command {} does not exist".format(prog))

else:
    try:
        filenames = os.listdir(in_path)
        for file in filenames:
            name, extension = os.path.splitext(file)
            if(extension == source):
                print("Processing: {}".format(file))
                inf = os.path.join(in_path, file)
                outf = os.path.join(out_path, name + new_ext)
                process = subprocess.Popen([prog, "-i", inf, "-b:a", arate, "-vn", outf], shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
                stdout,stderr = process.communicate()
                return_code = process.poll()
                print("stdout='{}'\nstderr='{}'\nreturn_code='{}'".format(stdout, stderr, return_code))
            else:
                print("Skipping: {}".format(file))

    except IOError as e:
        print("Error: file not found.")

If anyone has improvements on this, please let me know!

Answer 2

If in your initial example you are passing "shell=True", then this note in the docs is relevant:

The shell argument (which defaults to False) specifies whether to use the shell as the program to execute. If shell is True, it is recommended to pass args as a string rather than as a sequence

In which case:

subprocess.Popen(['ls -l'], shell=True)

results in:

total 1416
-rw-r--r--   1 me  AD\Domain Users      17 Nov  3 01:17 file1
drwxr-xr-x   7 me  AD\Domain Users     238 Jan 22 15:34 dir
-rw-r--r--   1 me  AD\Domain Users     368 Nov 14 22:39 file2.xml

Answer 3

If I understand your question correctly, then you want to convert arbitrary line from bash script to format that Popen accepts.

Python documentation has a nice (and well hidden) example on how to do this. From: http://docs.python.org/2/library/subprocess.html#popen-constructor

>>> import shlex, subprocess
>>> command_line = raw_input()
/bin/vikings -input eggs.txt -output "spam spam.txt" -cmd "echo '$MONEY'"
>>> args = shlex.split(command_line)
>>> print args
['/bin/vikings', '-input', 'eggs.txt', '-output', 'spam spam.txt', '-cmd', "echo '$MONEY'"]

---

Now we can apply information above in practice:

import shlex
command = "/usr/bin/ffmpeg -i 'in_path/in_file.ext' -b:a arate -vn 'out_path/out_file.ext'"
print command
# /usr/bin/ffmpeg -i 'in_path/in_file.ext' -b:a arate -vn 'out_path/out_file.ext'
popen_cmd = shlex.split(command)
print popen_cmd
# ['/usr/bin/ffmpeg', '-i', 'in_path/in_file.ext', '-b:a', 'arate', '-vn', 'out_path/out_file.ext']

Though it may be easier to just use os.system