The OP wrote:
Just in case someone find it interesting I am going to write out the lessons I learned. I think that the most important lesson I learnt is that common sense is a great tool to figure out what is a intern token in the .lex file and what is a suitable token to share with the .y file.
Since the term 'common sense' may be a bit ambiguous I post the following example:
ALPHA_NUMERIC [a-bA-B0-9] SQ_CHAR {SHARED}|{ALPHA_NUMERIC} SINGLE_QUOTED {SINGLE_QUOTE}{SQ_CHAR}{SQ_CHAR}*{SINGLE_QUOTE}
where
ALPHA_NUMERIC
is a good intern token (file.lex
) but is a bad token to share in the grammar file whereasSINGLE_QUOTED
may be a good token to share with the grammar(file.y
). I wrote 'may be' because it is very dependent of the specific grammar we are working on, in my concrete case it is a good token to share with the YACC file.What I did is to define as a token a regexp similar to the one @OGHaza advised me in
file.lex
and then I use it in the grammar itself (file.y
).