Why compgen needs escaping here?

Question

There is some interesting behavior of compgen -W when I use single quotes inside double quotes (look how the last argument and output differ from command to command):

$ compgen -W "a\'b1 a\'b2" -- "a'"
a'b1
a'b2
$ compgen -W "a\'b1 a\'b2" -- "a'b"
$ compgen -W "a\'b1 a\'b2" -- "a\'b"
a'b1
a'b2

Why does the third command need a \ before a ' to get completion options, but the first command doesn't? It took me hours to find out this fact while debugging my completion script. Is this a bug, or is there any explanation why it has to work this way?

Answer 1

It's because of the way compgen deals with quote characters and escape characters in the target. In effect, it takes a very literal view of what completion might mean.

Let's consider something like the first case, but where the possible words don't have any special characters:

ab1
ab2

In bash, you could type the first of these words in any of the following ways (amongst others):

ab1
a'b'1
a''b1
a'b1'

So both ab1 and ab2 are possible completions of a'. And that's what compgen returns.

Now, let's go back to the actual targets in the question:

a'b1
a'b2

The first of these could be typed in either of these ways (amongst others):

a\'b1
a''\'b1

but it cannot be typed like this:

a'\''b1

So all of the following can be completed to either of the targets:

a
a'
a\

But the following cannot be

a'b