How to insert an inline (heredoc maybe? ) python script into a bash stdin/stdout streaming pipeline

Question

I have recently been doing fair amount of work in python and would like to be able to use its features instead of shell/bash builtins / shell scripting.

So for a shell pipeline like this:

echo -e "Line One\nLine Two\nLine Three" | (cat<<-HERE | python
import sys
print 'stdout hi'
for line in sys.stdin.readlines():
  print ('stdout hi on line: %s\n' %line)
HERE
) | tee -a tee.out

All that is printed is "stdout hi"

What needs to be fixed here?

thanks!

Answer 1

It would be better if you explained what your goal is with this construction. Maybe it could be simplified?

The problem is with this script that the echo goes to the stdin of the encapsulating shell initiated by the (...) notation. But inside the shell stdin is redefined as the heredoc piped to python, so it reads the script from stdin, which is now comes from the heredoc pipe.

So you try something like this:

echo -e "Line One\nLine Two\nLine Three" |  python <(cat <<HERE
import sys
print "stdout hi"
for line in sys.stdin:
  print line.rstrip()
print "stdout hi"
HERE
)

Output:

stdout hi
Line One
Line Two
Line Three
stdout hi

Now the script is read from /dev/fd/<filehandle>, so stdin can be used by the echo's pipe.

SOLUTION #2

There is another solution. The script can be sent to python's stdin as here-is-the document, but then the input pipe has to be redirected to another file descriptor. For this an fdopen(3) like function has to be used in the script. I'm unfamiliar with python, so I show a perl example:

exec 10< <(echo -e "Line One\nLine Two\nLine Three")

perl <<'XXX'
print "stdout hi\n";
open($hin, "<&=", 10) or die;
while (<$hin>) { print $_; }
print "stdout hi\n";
XXX

Here the echo is redirected to the file handle 10, which is opened inside the script.

But the echo part can be removed (-1 fork), using an other heredoc:

exec 10<<XXX
Line One
Line Two
Line Three
XXX

MULTILINE SCIPRT

Or simply enter a multile script using the -c option:

echo -e "Line One\nLine Two\nLine Three"|python -c 'import sys
print "Stdout hi"
for line in sys.stdin:
  print line.rstrip()
print "Stdout hi"'