backslash in $ python -c "script"

Question

Today I wrote a Python script to process data and print them in customized format. It worked as expected under interactive Python. Then I moved on to try in Terminal to redirect the output to a text file, and encountered something unexpected.

The particular example I tried was to arrange a list of data in columns, and print them in LaTeX tabular format. It works more or less like:

foo(mylist, sep_col='\t', sep_row=r'\\ \hline')

When I tried to run the script in Terminal, I did this:

$ python -c "from foo import foo; foo(mylist, col_sep=' & ', row_sep=r' \\ \hline')" > results.txt

But it turned out that the \\ \hline part was shown as \ \hline, which is not the same as under interactive Python.

As I looked into the problem (or maybe the principles behind) more, I found that the behavior of the Python script processed by $ python -c is not the same under interactive Python.

Under interactive Python: >>> print r'\\' gives \\

However, in Terminal, $ python -c "print r'\\'" raises a SyntaxError:

  File "<string>", line 1
    print r'\'
             ^
SyntaxError: EOL while scanning string literal

Terminal seems to give \\ when the number of backslash is 3 or 4. Number 5 or 6 raises an error. 7 backslashes give \\\\, which means there is no way to give \\\ under this environment.

I've searched for this issue for quite a while, but it seems very few talks about $ python -c. Sorry in advance if I made a duplicate.

Answer 1

In terminal, saying:

$ python -c "print r'\\'"
  File "<string>", line 1
    print r'\'
             ^
SyntaxError: EOL while scanning string literal

results in an error because the shell comes into play. It translates \\ into \ (see the error message carefully).

Instead, say

$ python -c 'print r"\\"'
\\

when running from the shell, i.e. supply the command in single quotes and arguments to print in double quotes.

The equivalent of the command that you were executing in interactive mode would be:

$ python -c "print r'\\\\'"
\\

EDIT: This answer explains why a raw string cannot have an odd number of backslashes.

Moreover, this is also explained in the official FAQ.

Answer 2

Your shell interprets the backslashes inside the double quotes as well. You can use single quotes to avoid this; then you will have to change your single quotes in the python script to double quotes instead:

python -c 'from foo import foo; foo(mylist, col_sep=" & ", row_sep=r" \\ \hline")'

You could have used echo debugging in this case: Simply write echo before your shell command to see what is about to get executed:

echo python -c "print r'\\'"

shows:

python -c print r'\'

Answer 3

Your terminal does not recognise raw or regex strings, i.e. r'something' as anything special and all backslashes need to be escaped by backslashes in the terminal, so:

python -c "from foo import foo; foo(mylist, col_sep=' & ', row_sep=r' \\ \hline')" > results.tx

needs to be:

python -c "from foo import foo; foo(mylist, col_sep=' & ', row_sep=r' \\\\ \\hline');" > results.tx

and

foo(mylist, sep_col='\t', sep_row=r'\\ \hline')

would map to:

python -c "from foo import foo; foo(mylist, sep_col='\\t', sep_row=r'\\\\ \\hline');"