How do I stop multiple line output from command substitution from being concatenated in BASH script?

Question

I have text files in multiple subdirectories

some/path/one/file

some/path/two/file

some/path/three/file

some/path/foo/file

...

I need to be able to grep file for a match, but retain the individual line returns so that I can operate on each match individually with a loop later.

I have:

SITE=$(grep -w $1 some/path/*/file)

however, this just produces a really long string with everything just smooshed together in one line.

Ideally, This variable would contain out the value of * from the pathname on each line:

for example:

some/path/one/file

some/path/three/file

contain a line that has bar in it.

SITE=$(grep -w bar some/path/*/file)
echo $SITE

should look like this:

some/path/one/file:this is a line of text that has bar in it
some/path/three/file:this is a line of text that has bar in it

but instead, I get this:

some/path/one/file:this is a line of text that has bar in it some/path/three/file:this is a line of text that has bar in it

Is there a way to make this variable multi-line so that later I can do stuff like:

for i in `echo $SITE`; do
WHATEVER=$(echo $i | cut -d ':' -f 1 | cut -d '/' -f 3)
echo $WHATEVER
done

and have the output be:

one
three

Or am I going about this entirely the wrong way?

Answer 1

It is not the $() command substitution that smushes the output to one line; it is the echo of an unquoted variable. If you do echo "$SITE", you'll see the multi-line output.

Having said that, a common approach to reading line-by-line is the while read construct. You could do something like this:

while read line; do
    WHATEVER=$(echo "$i" | cut -d ':' -f 1 | cut -d '/' -f 3)
    echo $WHATEVER
done < <(grep -w bar some/path/*/file)

Note we can use a pipe instead of the <() process substitution on the last line, but the above is better if variables set within the while loop need to remain in scope outside of the while loop.

Of course, if this is all you need to do with the output, then the following is much more straightforward:

grep -w bar some/path/*/file | cut -d ':' -f 1 | cut -d '/' -f 3

Answer 2

If it's just for displaying, there's always echo -e "$SITE". The -e option leaves all tabs, spaces and newlines in place. Don't forget the quotes.

If you want to process each line later in the script, you could try the following:

echo -n "$SITE" | while read line ; do
  echo "$line"
done

Answer 3

IFS=$'\r\n' LINES=($(grep -w $1 some/path/*/file))

That will create an array of lines

You can then walk the array and treat each line individually ${LINES[i]}

See more here https://stackoverflow.com/a/11393884/652904