How to escape os.system() calls?

Question

When using os.system() it's often necessary to escape filenames and other arguments passed as parameters to commands. How can I do this? Preferably something that would work on multiple operating systems/shells but in particular for bash.

I'm currently doing the following, but am sure there must be a library function for this, or at least a more elegant/robust/efficient option:

def sh_escape(s):
   return s.replace("(","\\(").replace(")","\\)").replace(" ","\\ ")

os.system("cat %s | grep something | sort > %s" 
          % (sh_escape(in_filename), 
             sh_escape(out_filename)))

Edit: I've accepted the simple answer of using quotes, don't know why I didn't think of that; I guess because I came from Windows where ' and " behave a little differently.

Regarding security, I understand the concern, but, in this case, I'm interested in a quick and easy solution which os.system() provides, and the source of the strings is either not user-generated or at least entered by a trusted user (me).

Answer 1

This is what I use:

def shellquote(s):
    return "'" + s.replace("'", "'\\''") + "'"

The shell will always accept a quoted filename and remove the surrounding quotes before passing it to the program in question. Notably, this avoids problems with filenames that contain spaces or any other kind of nasty shell metacharacter.

Update: If you are using Python 3.3 or later, use shlex.quote instead of rolling your own.

Answer 2

shlex.quote() does what you want since python 3.

(Use pipes.quote to support both python 2 and python 3)

Answer 3

Perhaps you have a specific reason for using os.system(). But if not you should probably be using the subprocess module. You can specify the pipes directly and avoid using the shell.

The following is from PEP324:

Replacing shell pipe line
-------------------------

output=`dmesg | grep hda`
==>
p1 = Popen(["dmesg"], stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
output = p2.communicate()[0]

Answer 4

Maybe subprocess.list2cmdline is a better shot?

Answer 5

Note that pipes.quote is actually broken in Python 2.5 and Python 3.1 and not safe to use--It doesn't handle zero-length arguments.

>>> from pipes import quote
>>> args = ['arg1', '', 'arg3']
>>> print 'mycommand %s' % (' '.join(quote(arg) for arg in args))
mycommand arg1  arg3

See Python issue 7476; it has been fixed in Python 2.6 and 3.2 and newer.

Answer 6

I believe that os.system just invokes whatever command shell is configured for the user, so I don't think you can do it in a platform independent way. My command shell could be anything from bash, emacs, ruby, or even quake3. Some of these programs aren't expecting the kind of arguments you are passing to them and even if they did there is no guarantee they do their escaping the same way.

Answer 7

Notice: This is an answer for Python 2.7.x.

According to the source, pipes.quote() is a way to "Reliably quote a string as a single argument for /bin/sh". (Although it is deprecated since version 2.7 and finally exposed publicly in Python 3.3 as the shlex.quote() function.)

On the other hand, subprocess.list2cmdline() is a way to "Translate a sequence of arguments into a command line string, using the same rules as the MS C runtime".

Here we are, the platform independent way of quoting strings for command lines.

import sys
mswindows = (sys.platform == "win32")

if mswindows:
    from subprocess import list2cmdline
    quote_args = list2cmdline
else:
    # POSIX
    from pipes import quote

    def quote_args(seq):
        return ' '.join(quote(arg) for arg in seq)

Usage:

# Quote a single argument
print quote_args(['my argument'])

# Quote multiple arguments
my_args = ['This', 'is', 'my arguments']
print quote_args(my_args)

Answer 8

The function I use is:

def quote_argument(argument):
    return '"%s"' % (
        argument
        .replace('\\', '\\\\')
        .replace('"', '\\"')
        .replace('$', '\\$')
        .replace('`', '\\`')
    )

that is: I always enclose the argument in double quotes, and then backslash-quote the only characters special inside double quotes.

Answer 9

If you do use the system command, I would try and whitelist what goes into the os.system() call.. For example..

clean_user_input re.sub("[^a-zA-Z]", "", user_input)
os.system("ls %s" % (clean_user_input))

The subprocess module is a better option, and I would recommend trying to avoid using anything like os.system/subprocess wherever possible.

Answer 10

The real answer is: Don't use os.system() in the first place. Use subprocess.call instead and supply the unescaped arguments.