Flex lexer start condition not being recognized

Question

I have a scanner that maintains two exclusive states (WORD and DEFN)

%option noyywrap
%s WORD
%s DEFN
%%

%{
    BEGIN(WORD);
%}

<WORD>{
"|"          { BEGIN(DEFN); return WS_PIPE; }
}

<DEFN>{
[^;]+        { printf("ds: %s\n", yytext); return WD_STRING; }
";"          { return WD_SEMICOLON; }
}

\n|.         { printf("U: %s\n", yytext); }

%%

But with the simple input "| some text;", when the pipe is parsed the state is not being changed, so the parsing of "some text;" fails.

Answer 1

The state is certainly being changed to DEFN when a | is encountered in state WORD. However, the next time yylex is called (to get the token following the pipe), the state is reset to WORD by the block

%{
    BEGIN(WORD);
%}

From the flex manual (emphasis added):

In the rules section, any indented or %{ %} enclosed text appearing before the first rule may be used to declare variables which are local to the scanning routine and (after the declarations) code which is to be executed whenever the scanning routine is entered.

You're really better off using the INITIAL start condition to represent the, well, initial start condition.