How do I capture the line number of a syntax error in generated parser

Question

All of the tokens I pass to Lemon are structures that have line number information attached.

Look at the syntax_error definition below

%name SinkParser
%token_prefix SINKPARSER_TOKEN_

%token_type {SinkParserNode*}

%extra_argument { SinkParserContext *parser_context }
%syntax_error {
  SinkParser_SyntaxError(parser_context);
}
%token_destructor { SinkParserNode_Destruct($$); }

However I don't see how to report what file and line the error was. I have the information on all tokens but how do I access the appropriate one in the syntax_error handler?

Do I need to separately keep track of my place in the tokenizer and peek at it in the syntax error? I realize that the tokenizer will probably be ahead of the parser by the time it hits syntax error.

Answer 1

LALR(1) parsers never consume (shift) a token which cannot match a production, although they may perform erroneous reductions. So the last token which you provided to the parser is the token which triggered the syntax error (the last token is the lookahead token; if it cannot be shifted, that's a syntax error). It would probably be useful to know where the previous token was as well, since the two tokens may be separated by an arbitrary amount of whitespace and comments. (For example, if a command is missing a terminator and is followed by a long comment, the error token might be quite some distance forward.)

Answer 2

Yes, you need to keep track in the tokenizer. Either as a "global" variable, or as an attribute of the token.