Get array of tokens

Question

Is there any way to store the tokens that have been passed on to the parser after the lexing process (preferably in order in a data structure like an array)? Furthermore, is it possible to convert these tokens to String literals? If so, some guidance regarding how to go about this would be highly appreciated.

Answer 1

Yes and yes.

Each token contains a pointer to the next token, so the tokens form a linked list. If you want them in an array, you can easily traverse the list and put the tokens into an array.

Each token has an image field that contains the character sequence that the token represents.

See the JavaCC FAQ question 5.2 for more detail. http://www.engr.mun.ca/~theo/JavaCC-FAQ/javacc-faq-moz.htm#tth_sEc5.2