ANTLR4 Flattening a ParserRuleContext Tree into an Array

Question

How to flatten a ParserRuleContext with subtrees into an array of tokens? The ParserRuleContext.getTokens(int ttype) looks good. but what is ttype? Is it token type? What value to use if I want to include all token types?

Answer 1

ParserRuleContext.getTokens(int ttype) only retrieves certain child nodes of a parent: it does not recursively go into the parent-tree.

However, it is easy enough to write yourself:

/**
 * Retrieves all Tokens from the {@code tree} in an in-order sequence.
 *
 * @param tree
 *         the parse tee to get all tokens from.
 *
 * @return all Tokens from the {@code tree} in an in-order sequence.
 */
public static List<Token> getFlatTokenList(ParseTree tree) {
    List<Token> tokens = new ArrayList<Token>();
    inOrderTraversal(tokens, tree);
    return tokens;
}

/**
 * Makes an in-order traversal over {@code parent} (recursively) collecting
 * all Tokens of the terminal nodes it encounters.
 *
 * @param tokens
 *         the list of tokens.
 * @param parent
 *         the current parent node to inspect for terminal nodes.
 */
private static void inOrderTraversal(List<Token> tokens, ParseTree parent) {

    // Iterate over all child nodes of `parent`.
    for (int i = 0; i < parent.getChildCount(); i++) {

        // Get the i-th child node of `parent`.
        ParseTree child = parent.getChild(i);

        if (child instanceof TerminalNode) {
            // We found a leaf/terminal, add its Token to our list.
            TerminalNode node = (TerminalNode) child;
            tokens.add(node.getSymbol());
        }
        else {
            // No leaf/terminal node, recursively call this method.
            inOrderTraversal(tokens, child);
        }
    }
}