Pipe Find command “stderr” to a command

Question

Hi I have a peculiar problem and I'm trying hard to find (pun intended) a solution for it.

$> find ./subdirectory -type f 2>>error.log

I get an error, something like, "find: ./subdirectory/noidea: Permission denied" from this command and this will be redirected to error.log.

Is there any way I can pipe the stderr to another command before the redirection to error.log?

I want to be able to do something like

$> find ./subdirectory -type f 2 | sed "s#\(.*\)#${PWD}\1#" >> error.log

where I want to pipe only the stderr to the sed command and get the whole path of the find command error.

I know piping doesn't work here and is probably not the right way to go about.

My problem is I need both the stdout and stderr and the both have to be processed through different things simultaneously.

EDIT: Ok. A slight modification to my problem.

Now, I have a shell script, solve_problem.sh

In this shell script, I have the following code

ErrorFile="error.log"
for directories in `find ./subdirectory -type f 2>> $ErrorFile`
do
    field1=`echo $directories | cut -d / -f2`
    field2=`echo $directories | cut -d / -f3`
done

Same problem but inside a shell script. The "find: ./subdirectory/noidea: Permission denied" error should go into $ErrorFile and stdout should get assigned to the variable $directories.

Answer 1

Pipe stderr and stdout simultaneously - idea taken from this post:

(find /boot | sed s'/^/STDOUT:/' ) 3>&1 1>&2 2>&3 | sed 's/^/STDERR:/'

Sample output:

STDOUT:/boot/grub/usb_keyboard.mod
STDERR:find: `/boot/lost+found': Brak dostępu

Bash redirections like 3>&1 1>&2 2>&3 swaps stderr and stdout.

I would modify your sample script to look like this:

#!/bin/bash
ErrorFile="error.log"
(find ./subdirectory -type f 3>&1 1>&2 2>&3 | sed "s#^#${PWD}: #" >> $ErrorFile) 3>&1 1>&2 2>&3 | while read line; do
    field1=$(echo "$line" | cut -d / -f2)
    ...
done

Notice that I swapped stdout & stderr twice.

Small additional comment - look at -printf option in find manual page. It might be useful to you.

Answer 2

If you need to redirect stderr to stdout so that the following command in the pipe gets it as its input, then you can use 2>&1.

For more information, please have a look at the all about redirection how-to.

Edit: Even if you need to edit pass stdout further in the pipe, you can use sed to filter error messages and write them to a file:

$ find . -type f 2>&1 | sed '/^find:/{
> w error.log
> d
> }'

In this example:

• in find command stderr is redirected to stdout
• in sed command errors from find that match a regular expression are written to a file (w error.log) and removed from output (d).
• any command in the pipeline following the pipeline will received the output from find.

Note: This will work as lon as all the error messages from find start with find:. Otherwise, the regular expression in sed should be modified to properly match all cases.

Answer 3

You can try this (on bash), which appears to work:

find ./subdirectory -type f 2> >(sed "s#\(.*\)#${PWD}\1#" >> error.log)

This does the following:

1. 2> redirects stderr to
2. >(...) a process substitution (running sed, which appends to error.log)