How do I use regular expression capturing groups with JFlex?

Question

Although this question is about JFlex, it probably applies to other scanner generators such as lex, flex as well.

If I have some rule, how can I create a capturing group in part of that rule and use the result of that captured group as an argument to the code that gets called upon the rule matching?

For example, let's say I had a simple rule to match an SGML tag:

"<"[a-zA-Z]+">"    {return new Token(Type.OPEN_TAG);}

How could I capture the inner character part([a-zA-Z]+) and use it as an argument in my Token constructor?

Edit: I'm aware I could simply use yytext() to get the whole matched value and then separate the parts elsewhere in code, but that seems like it would be making things more complicated than they need to be.

Answer 1

Scanner generators generally don't support capturing groups, and to be honest, I have never seen a valid need for them in a scanner generator. Most things you would normally us the capturing groups for in other RegEx engines are better handled in the parser or by a simple piece of code in the action.

Something like the following should probably work.

"<"[a-zA-Z]+">"    {
                     String matchedText = yytext();
                     String label = matchedText.substring(1, matchedText.length() - 1);
                     return new Token(Type.OPEN_TAG, label);
                   }

---

Implementing group capturing tends to interfere with many of the optimisations performed by the scanner generator to reduce the size of the transition table. I have never used JFlex but I seem to remember something about flex supporting some limited form of backtracking and look ahead/behind, but would then issue warnings about performance if used.