Making bison/flex parser reentrant with integral YYSTYPE

Question

I'm having trouble following the steps to make my bison/flex parser reentrant with a minimum amount of fuss.

The problem appears to be in the lexer. Since everything parser is re-entrant, I can no longer assign yylval directly. Instead, according to the Flex manual, I have to call this function:

void yyset_lval ( YYSTYPE * yylvalp , yyscan_t scanner );

But the problem is, YYSTYPE is an integral type. It isn't a dynamically allocated value, and it isn't an lvalue at all, so I can't pass a pointer to it! Am I missing something, and if not, how am I supposed to set yylvalue? I've never had this problem with a non-reentrant parser!

Update: Ok, I think I may have gotten past the integral type issue. Now the parser appears to crash because no input is set up, and I am not finding any documentation on how to tell the lexer where to read input from.

Flex file:

%{
#define YYSTYPE Token // Token is a typedef to an integral type
// following two lines added per suggestion from Chris Dodd
#define YYPARSE_PARAM yyscan_t scanner
#define YYLEX_PARAM scanner

%}
%option reentrant
%option bison-bridge

// ...

{D}+{IS}?               {
    *yylval = Tokenint(strtol(yytext,0,10));
     return(CONSTANT); 
}

Bison file:

%{
#define YYSTYPE Token
%define pure-parser
%lex-param {void * scanner}
%parse-param {void * scanner}
%token CONSTANT
// ...

Getting a crash at this line in the parser:

*yy_cp = yyg->yy_hold_char;

I suspect I may not be initializing everything correctly for the parse. The manuals are very confusing on this subject, often referencing types or variable without explaining where they come from or why they are used. I am invoking the parse this way:

void * scanner;
yylex_init(&scanner);
yyparse(scanner); // Note: was &scanner, this is wrong and will crash!

From reading through the souce code, yylex_init is allocating a struct yyguts_t and zeroing it out. When I reach the crash, yy_cp is a null pointer, in fact the entire struct yyguts_t is still zero-ed out! So I suspect I am not initializing something properly, and that is resulting in it not knowing where to get input from.

Final edit: I'm going to mark this question as closed as my original question was answered, and re-open the new problem as a new question.

Answer 1

Normally to make a reentrant parser/scanner, you'll use %option bison-bridge in your .l file and %define api.pure in your .y file. This makes yylval an argument to yylex instead of a global variable. In addition, you need have:

%{
#define YYPARSE_PARAM yyscan_t scanner
#define YYLEX_PARAM scanner
%}

in the top of your .y file to pass the extra scanner parameter through your parser to your lexer.

Now if YYSTYPE is int (you have no %union or declaration of YYSTYPE in your .y file), then you'd set the token value in your .l by just saying *yylval = whatever;