Your problem is in your lexical analyzer (independently of whether there's a problem in your grammar — I've not analyzed the grammar, because I hit on the problem in the lexical analyzer first, and it was sufficient to prevent the grammar working).
I added a test main()
to lexer.l
:
%%
YYSTYPE yylval;
int main(void)
{
int token;
while ((token = yylex()) != 0)
printf("Token: %d (%s)\n", token, yytext);
return 0;
}
I then ran it on your sample code to see whether the token stream is generated correctly. The output I got is:
TOKEN_BEGIN
Token: 258 (begin)
WhiteSpaces
COMMENT
NewLine
WhiteSpaces
TOKEN_IDENTIFIER
Token: 265 (x)
WhiteSpaces
yytext = =
Token: 61 (=)
WhiteSpaces
TOKEN_VALUE
Token: 264 (56)
yytext = ;
Token: 59 (;)
NewLine
NewLine
WhiteSpaces
While
Token: 260 (while)
WhiteSpaces
TOKEN_IDENTIFIER
Token: 265 (x)
WhiteSpaces
TOKEN_OPERATOR
Token: 263 (>)
WhiteSpaces
TOKEN_VALUE
Token: 264 (0)
WhiteSpaces
DO
Token: 261 (do)
NewLine
WhiteSpaces
TOKEN_BEGIN
Token: 258 (begin)
NewLine
WhiteSpaces
POINT_START
Token: 262 (point)
WhiteSpaces
POINT_DEFINITIONS 15.6
Token: 266 (15.6)
WhiteSpaces
POINT_DEFINITIONS 78.96
Token: 266 (78.96)
;
WhiteSpaces
end;
end;
As you can see, the last token returned to the main program is the 78.96.
When you recognize a POINT, you start the state POINT_DEFINITIONS. However, once in that state, you're in that state for ever; you never go back to the INITIAL state. You probably need to add a rule to the POINT_DEFINITIONS start state that recognizes a semicolon and executes BEGIN INITIAL;
:
<POINT_DEFINITIONS>{Semicolon} {
printf("Semicolon in POINT_DEFINITION state\n");
BEGIN INITIAL;
return *yytext;
}
With this in place, the tail end of the output is:
...
TOKEN_BEGIN
Token: 258 (begin)
NewLine
WhiteSpaces
POINT_START
Token: 262 (point)
WhiteSpaces
POINT_DEFINITIONS 15.6
Token: 266 (15.6)
WhiteSpaces
POINT_DEFINITIONS 78.96
Token: 266 (78.96)
Semicolon in POINT_DEFINITION state
Token: 59 (;)
NewLine
WhiteSpaces
TOKEN_END
Token: 259 (end)
punctuation: yytext = ;
Token: 59 (;)
NewLine
TOKEN_END
Token: 259 (end)
punctuation: yytext = ;
Token: 59 (;)
NewLine