The lexical analysis part: analyse ?: as an operator instead of separator

Question

My target is to implement a very simple lexical analysis of the C language, such as when you read an expression ab=3, then It will be analysed as:

ID<ab> OP<=> LIT<8>   (op will be "operator",LIT will be "literal"). 

There is also an situation when you read an expression a ? b :c. normally it will be analysed as

ID<a> SEP<?> ID<b> SEP<:> ID<c>". 

but as we know It actually is the ternary operator. So the analysis listed above is not correct.

Now I just want to refer to a compiler such as gcc, g++ and check how do these compilers implement the lexical analysis? Anyone who can give me an suggestion ?

Answer 1

What makes that lexer output incorrect? You are thinking too far ahead and trying to solve parse problems in the lexer. If you really like, you could change ? to be an operator (as it should be in this case), but ultimately it doesn't matter, just parse using the terminals you have defined as tokens.