RegExp Wrong Special Character Result On Group Reference

Question

What is my mistake ?

>>> import re
>>> re.sub("(\[*?)","\1","x[[[y")
'\x01x\x01[\x01[\x01[\x01y\x01'

How can I do ?

Answer 1

Use raw strings for regex:

>>> import re
>>> re.sub(r'(\[*?)', r'\1', 'x[[[y')
'x[[[y'

When you write '\1', \1 is being interpreted as an escape sequence. '\\1' is the proper way to write it with normal strings, which is equivalent to r'\1'.

Answer 2

\[*? matches zero or more left square brackets, reluctantly. In other words, it matches nothing, every time, no matter where you apply it.

Quantifiers are normally greedy, which means they match as much as they can while still allowing the regex as a whole to match. That's what "backtracking" means most of the time: greedy quantifiers giving back characters they gobbled up so the rest of the regex can match what it needs to.

Reluctant quantifiers do just the opposite: they consume the minimum number of characters at first, then they take more only if that's necessary to achieve an overall match. But in your case it's never necessary; \[*? is the whole regex, so it's content to match an empty string. It never makes sense to have a reluctant quantifier as the last thing in a regex.

Just to clear up any possible confusion, the brackets in your original string are not being captured and reinserted, they're being left there, just like the x and y are. If you had used raw strings as @Blender suggested (correctly--that was another error), your output would have been the original string:

>>> re.sub(r"(\[*?)",r"\1","x[[[y")
'x[[[y'

It's matching an empty string at every character boundary (i.e., the beginning and end, and between each pair of characters), and reinserting them--total effect, none. I don't know what you were trying to accomplish, but if you wanted to replace the three brackets with one, you should have used + instead of *?, and you should have put it outside the capturing group:

>>> re.sub(r"(\[)+",r"\1","x[[[y")
'x[y'