Two quick questions about flex / C

Question

I would like to use this idiom :

    yy_scan_string(line);
    int i;
    while ((i = yylex()))
        ....

where these two functions are define in the flex generated lex.yy.c in my main C file. So far, I am

 #including "lex.yy.c"

but it seems fishy. How do I do that the correct C way? Secondly, I would like the last line of my .l file,

.                   {   return WORD; }

to no longer return a "WORD" token, but rather to return its input. For exemple (it is a smallish linux shell)

 ls > ls.txt

Currently returns 2 WORD tokens, a GREATER token, and 6 WORD tokens, when I would like a return of "ls" GREATER "ls.txt". Of course yylex() can only return one type, so what is the accepted way to obtain the desired result?

Thanks.

Answer 1

You can tell flex to generate a header file as well as the C source file, using the --header-file=<filename> command line option, or by including %option header-file="<filename>" in the flex source. I typically invoke flex with:

flex --header-file=file.h -o file.c file.l

(Actually, I use make rules to generate a command like that, but that's the idea.) Then you can #include "file.h" in any source file which needs to invoke a flex function.

Normally, yylex returns the token type (an integer). The global variable yytext contains a pointer to the token string itself, which is probably sufficient for your purposes. However, please read "A Note About yytext And Memory" in the flex manual. (Summary: if you need to save the value of yytext, you must make a copy of it; strdup is recommended. Don't forget to free the copy when you don't need it anymore.)

Sometimes, the token string itself is not exactly what you want as a semantic value. By convention, flex actions place the semantic value of the token in the global yylval, which is where bison-generated parsers will look for it. However, yylval is not declared anywhere by flex-generated code, so you need to include a declaration yourself, both in the flex-generated code and in any source file which includes it. (If you use bison to generate your parser, bison will generate this declaration and put it in the header file it generates.)