Simple looking question with a complicated answer. Most of the issue is with the shell; it is only partly a problem with sed
. (In other words, you could use a number of different commands instead of sed
and would run into similar issues.)
Note that most commands documented with an option letter and a separate argument string will also work when the argument string is attached to the option. For example:
sort -t :
sort -t:
Both of these give the value :
to the -t
option. Similarly with sed
and the -e
option. That is, you can write either of these:
sed -n -e /match/p
sed -n -e/match/p
Let's look at the one of the working sed
commands you wrote:
$ s="-e s/a/b/"
$ ls | sed $s
What the sed
command is passed here is two arguments (after it's command name):
-e
s/a/b/
This is a perfectly fine set of arguments for sed
. What went wrong with the first one, then?
$ s="-e 's/^ *//' -e 's/ *$//'"
$ ls | sed $s
Well, this time, the sed
command was passed 6 arguments:
-e
's/^
*//'
-e
's/
*$//'
You can use the al
command (argument list — print each argument on its own line; it is described and implemented at the bottom of this answer) to see how arguments are presented to sed
. Simply type al
in place of sed
in the examples.
Now, the -e
option should be followed by a valid sed
command, but 's/^
is not a valid command; the quote '
is not a valid sed
command. When you type the command at the shell prompt, the shell processes the single quote and removes it, so sed
does not normally see it, but that happens before shell variables are expanded.
Why, then, does the eval
work:
$ s="-e 's/^ *//' -e 's/ *$//'"
$ ls | eval sed $s
The eval
re-evaluates the command line. It sees:
eval sed -e 's/$ *//' -e 's/ *$//'
and goes through the full evaluation process. It removes the single quotes after grouping the characters, so sed
sees:
-e
s/$ *//
-e
s/ *$//
which is all completely valid sed
scripting.
One of your tests was:
$ s="-e s/^ *//"
$ ls | sed $s
And this failed because sed
was given the arguments:
-e
s/^
*//
The first is not a valid substitute command, and the second is unlikely to be a valid file name. Interestingly, you could rescue this by putting double quotes around the $s
, as in:
$ s="-e s/^ *//"
$ ls | sed "$s"
Now sed
gets a single argument:
-e s/^ *//
but the -e
can have the command attached, and leading spaces on commands are ignored, so this is all valid. You can't do that with your first attempt, though:
$ s="-e 's/^ *//' -e 's/ *$//'"
$ ls | sed "$s"
Now you get told about the '
not being recognized. You could, however, have used:
$ s="-e s/^ *//; s/ *$//"
$ ls | sed "$s"
Again, sed
sees a single argument, and there are two semicolon-separated sed
commands in the argument to the -e
option.
You can ring the variations from here. I find the al
command very useful; it quite often helps me understand where something is going wrong.
Source for al
— argument list
#include <stdio.h>
int main(int argc, char **argv)
{
while (*++argv)
puts(*argv);
return 0;
}
This is one of the smallest useful C programs you can write ('hello world' is one line shorter, but it isn't useful for much beyond demonstrating how to compile and run a program). It lists each of its arguments on a line on its own. You can also simulate it in bash
and other related shells with the printf
command:
printf "%s\n" "$@"
Wrap it as a function:
al()
{
printf "%s\n" "$@"
}