Python module to shellquote/unshellquote? [duplicate]

Question

This question already has an answer here:

• How to escape os.system() calls? 10 answers

Is there anything in the Python standard library that will properly parse/unparse strings for using in shell commands? I'm looking for the python analog to perl's String::ShellQuote::shell_quote:

$ print String::ShellQuote::shell_quote("hello", "stack", "overflow's", "quite", "cool")
hello stack 'overflow'\''s' quite cool

And, even more importantly, something which will work in the reverse direction (take a string and decompose it into a list).

Answer 1

pipes.quote is now shlex.quote in python 3. It is easy enough to use that piece of code.

https://github.com/python/cpython/blob/master/Lib/shlex.py#L281

That version handles zero-length argument correctly.

Answer 2

Looks like

try:  # py3
    from shlex import quote
except ImportError:  # py2
    from pipes import quote

quote("hello stack overflow's quite cool")
>>> '"hello stack overflow\'s quite cool"'

gets me far enough.

Answer 3

I'm pretty sure that pipes.quote is broken, and should not be used, because it does not handle zero-length arguments correctly:

>>> from pipes import quote
>>> args = ['arg1', '', 'arg3']
>>> print 'mycommand %s' % (' '.join(quote(arg) for arg in args))
mycommand arg1  arg3

I believe the result should be something like

mycommand arg1 '' arg3

Answer 4

For shell quoting, this works: I've rigorously tested it on Posix. [I'm assuming that the list2cmdline function supplied by Python works as advertised on Windows]

# shell.py
import os
if os.name == 'nt':
    from subprocess import list2cmdline

    def quote(arg):
        return list2cmdline([arg])[0]
else:
    import re
    _quote_pos = re.compile('(?=[^-0-9a-zA-Z_./\n])')

    def quote(arg):
        r"""
        >>> quote('\t')
        '\\\t'
        >>> quote('foo bar')
        'foo\\ bar'
        """
        # This is the logic emacs uses
        if arg:
            return _quote_pos.sub('\\\\', arg).replace('\n',"'\n'")
        else:
            return "''"

    def list2cmdline(args):
        return ' '.join([ quote(a) for a in args ])

The tests are here, if anyone cares.

Answer 5

To unquote, try shlex.split()

Answer 6

You should never have to shell quote. The correct way to do a command is to not do shell quoting and instead use subprocess.call or subprocess.Popen, and pass a list of unquoted arguments. This is immune to shell expansion.

i.e.

subprocess.Popen(['echo', '"', '$foo'], shell=False)

If you want to unquote shell quoted data, you can use shlex.shlex like this:

list(shlex.shlex("hello stack 'overflow'\''s' quite cool"))

Answer 7

The standard library module subprocess has the list2cmdline function which does this, albeit according to Microsoft rules so I am not sure how reliable it works in Unix-like environments for more complicated command lines.

Answer 8

The quotefunction is available for quite some time (Python 2.7?) -- the major drawback is it moved from pipe module to shlex between 3.2 and 3.3.

You have to be prepared to handle both cases while importing that function:

try:
    from shlex import quote
except ImportError:
    from pipes import quote