Subprocess.check_output with grep: exit-status 2

Question

I've used subprocess before without any problems, for some reason when I try it with grep:

grepOut = subprocess.check_output("grep 'hello' tmp", shell=True)

I get the following error:

File "/usr/lib/python2.7/subprocess.py", line 544, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['grep', "'hello'", 'tmp']' returned non-zero exit status 2

I don't get any errors by typing the command directly in a terminal.

EDIT: see clemej's answer for the explanation

Answer 1

You're using the wrong arguments when shell=True.

See https://docs.python.org/2/library/subprocess.html

When you're using shell=True, the first argument isn't a list of string arguments, but the command as a string:

grepOut = subprocess.check_output("grep 'hello' tmp", shell=True)

should work.

You only need to use the list form when not specifying shell=True, so alternatively:

grepOut = subprocess.check_output(['grep', "'hello'", 'tmp'])

should also work.

Answer 2

One thing you could try is to read from the standard error stream to see what is going wrong. You could also just use call() and read the output from stdout, thereby ignoring the return code.

Answer 3

The correct command is: out = subprocess.check_output(['grep', 'hello', 'tmp']).

Note: no shell=True, no quotes inside quotes.

grep returns exit status 2 if an error happens. In this case the original code in your question was equivalent to calling grep without any arguments that is incorrect: grep expects a pattern as a mandatory argument.