READING stderr from within Awk

Question

I want to keep SSH debug info separate (and logged) from other input. However, if I simply redirect stderr to a log file, I risk combining output from SSH and output from the remote process on the host machine (that might send something to stderr):

$ ssh -v somemachine 2> file.log

So, I want to filter out only those lines that match "debug1":

$ ssh -v somemachine | awk '/debug1/ {print > "file.log"; next} {print}'

Good so far, BUT ssh's debug output goes to stderr. So...

$ ssh -v somemachine 2>& | awk '/debug1/ {print > "file.log"; next} {print}'

Foiled again! I don't want to mix stdout and stderr. BAD!

What does a kid like me do? I was about to go the route of named pipes or some such wildness, but really, all I need to know is how to get awk to match patterns from stderr ONLY.

Answer 1

Awk can't actually see the input from stderr - the pipe only pipes stdout. You're going to have to muck about with redirection in order to do what you want. The answer is really simple if you don't care about what's on stdout:

(ssh -v hostname somecommand > /dev/null) 2>&1 | awk '/debug1/ ...'

If you care about what's on stdout, you'll have to do something more complex. I believe that this will work:

((ssh -v hostname somecommand 1>&3) 2>&1 | awk '/debug1/ ...' 1>&2) 3>&1

In order, those redirects:

• redirect the original stdout aside to file descriptor 3
• redirect stderr to stdout for awk to see
• redirect stdout back to stderr (where it originally was)
• redirect file descriptor 3 back to stdout (where it originally was)

P.S. This solution is sh/bash-specific. Jonathan Leffler says it should work with ksh as well. If you're using a different shell, you can try to find the syntax to do the same redirects with it - but if that different shell is csh/tcsh, you're probably just screwed. The cshells are notoriously bad at handling stdout/stderr.

Answer 2

Give this a try (no subshells):

{ ssh -v somemachine 2>&1 1>&3- | awk '/debug1/ {print > "file.log"; next} {print}'; } 3>&1

or

{ ssh -v somemachine 2>&1 1>&3- | grep debug1 > file.log; } 3>&1

See this.

Answer 3

Thanks a lot..! I used ls with awk and was not able to redirect.

I used:

FILES=`ls -lrt *.txt | awk '{print $9}' 2>> /dev/null`

Now:

FILES=`ls -lrt *.txt 2>> /dev/null | awk '{print $9}'`