In an attribute value template, curly braces are special syntax indicating an XPath expression to be evaluated. If you want literal curly braces, you have to escape them by doubling:
An attribute value template consists of an alternating sequence of fixed parts and variable parts. A variable part consists of an XPath expression enclosed in curly brackets ({}). A fixed part may contain any characters, except that a left curly bracket must be written as {{ and a right curly bracket must be written as }}.
Note:
An expression within a variable part may contain an unescaped curly bracket within a StringLiteral XP or within a comment.
Not all attributes are AVTs, but the regex attribute of analyze-string is:
Note:
Because the regex attribute is an attribute value template, curly brackets within the regular expression must be doubled. For example, to match a sequence of one to five characters, write regex=".{{1,5}}". For regular expressions containing many curly brackets it may be more convenient to use a notation such as regex="{'[0-9]{1,5}[a-z]{3}[0-9]{1,2}'}", or to use a variable.
(Emphasis added, in both quotes.)