what does ‘Consume’ mean in javacc?

Question

Consume always appears in such places:(in the generated *Parser.java file of *.jj file)

 jj_consume_token(0);

 jj_consume_token(-1);

what do 0 and -1 mean in the code above?

private Token jj_consume_token(int kind) throws ParseException {
Token oldToken;
if ((oldToken = token).next != null) token = token.next;
else token = token.next = token_source.getNextToken();
jj_ntk = -1;
if (token.kind == kind) {
  jj_gen++;
  if (++jj_gc > 100) {
    jj_gc = 0;
    for (int i = 0; i < jj_2_rtns.length; i++) {
      JJCalls c = jj_2_rtns[i];
      while (c != null) {
        if (c.gen < jj_gen) c.first = null;
        c = c.next;
      }
    }
  }
  trace_token(token, "");
  return token;
}
token = oldToken;
jj_kind = kind;
throw generateParseException();
}

what does the function's return-Token mean?

in all , what does ‘Consume’ mean in javacc?

Answer 1

Conceptually the input stream is a sequence of regular (i.e. not special) tokens. One of these tokens is the "current token". Tokens prior to the current token may be garbage --i.e the generated code keeps no pointer to them--, tokens after the current token may not have been constructed yet. The current token is the one that is used by the parser to make most of its decisions. E.g. a production like

void A() : {} {  <X>  <Y> <Z> | <Y> <Z> | <Z> }

will be converted to a routine that does a switch based on the kind of the current token.

void A() {
    switch( <<the kind of the current token>> ) {
    case X: jj_consume_token(X) ; jj_consume_token(Y) ; jj_consume_token(Z) ; return ;
    case Y: jj_consume_token(Y) ; jj_consume_token(Z) ; return ;
    case Z: jj_consume_token(Z) ; return ; }

A pointer to the current token is held in field token. The jj_consume_token method replaces this token with the next one in the input. Basically what jj_consume_token does is this

if( token.next == null ) token.next = <<construct the next token>> ;
token = token.next ;
if( token.kind != kind) <<throw an exception>> ;
else return token ;

The kind parameter is used to indicate the kind of token expected.