How to Manipulate tree grammar

Question

PARSER GRAMMAR
protocol.g

grammar protocol; 

options {
  language = Java;
  output = AST;
  ASTLabelType=CommonTree;
}

tokens{ 
TRANSITIONS;
PAIR;
}

@header {
package com.javadude.antlr3.x.tutorial;
}

@lexer::header {
  package com.javadude.antlr3.x.tutorial;
}

parse
 : transitions EOF!
   {
     CommonTree root = $transitions.tree;

     int count = root.getChildCount();

     Tree child1 = root.getChild(0);
     Tree child2 = root.getChild(1);
     Tree child3 = root.getChild(2);
     Tree child4 = root.getChild(3);

     System.out.println("root=" + root.getToken().getText() + " has " + count + " child nodes:");
     System.out.println(" - child1=" + child1.toStringTree());
     System.out.println(" - child2=" + child2.toStringTree());
     System.out.println(" - child3=" + child3.toStringTree());
     System.out.println(" - child4=" + child4.toStringTree());
   }
 ;
transitions
 : 'transitions' '=' INT pair+ ';' -> ^(TRANSITIONS INT pair+)
 ;
pair
 : '(' INT ',' INT ')' -> ^(PAIR INT INT)
 ;

INT 
    : ('0'..'9')+;
WHITESPACE
    : ('\t' | ' ' | '\r' | '\n' | '\u000C')+ {$channel = HIDDEN;};

TREE GRAMMAR
protocolWalker.g

tree grammar protocolWalker;

options {
  language = Java;
  tokenVocab = protocol;   
  ASTLabelType = CommonTree;
}


@header {
package com.javadude.antlr3.x.tutorial;
}

transitions
 : ^(TRANSITIONS INT pair+) 
 {
 System.out.println("transitions=" + $INT.text);
 }
 ;

pair
 : ^(PAIR a=INT b=INT) 
 {
 System.out.println("pair=" + $a.text + ", " + $b.text);

 }
 ;

JAVA TEST RIG
Protocoltest.java

package com.javadude.antlr3.x.tutorial;
import org.antlr.runtime.*;
import org.antlr.runtime.tree.CommonTree;
import org.antlr.runtime.tree.CommonTreeNodeStream;
public class Protocoltest {

    /**
     * @param args
     */
    public static void main(String[] args) throws Exception {
        //create input stream from standard input
        ANTLRInputStream input = new ANTLRInputStream(System.in);
        //create a lexer attached to that input stream
        protocolLexer lexer = new protocolLexer(input);
        //create a stream of tokens pulled from the lexer
        CommonTokenStream tokens = new CommonTokenStream(lexer);

        //create a parser attached to teh token stream
        protocolParser parser = new protocolParser(tokens);
        //invoke the program rule in get return value
        protocolParser.parse_return r =parser.parse();

        CommonTree t = (CommonTree)r.getTree();
        //output the extracted tree to the console
        System.out.println("\nAST is: " + t.toStringTree());

        //walk resulting tree; create treenode stream first
        CommonTreeNodeStream nodes = new CommonTreeNodeStream(t);
        //AST nodes have payloads that point into token stream
        nodes.setTokenStream(tokens);

        //create a tree walker attached to the nodes stream
        protocolWalker walker = new protocolWalker(nodes);

        //invoke the start symbol, rule parse
        walker.transitions();
        }
}

INPUT

transitions = 3(5,0) (5,1) (5,2);

OUTPUT

root=TRANSITIONS has 4 child nodes:
 - child1=3
 - child2=(PAIR 5 0)
 - child3=(PAIR 5 1)
 - child4=(PAIR 5 2)

AST is: (TRANSITIONS 3 (PAIR 5 0) (PAIR 5 1) (PAIR 5 2))
pair=5, 0
pair=5, 1
pair=5, 2
transitions=3  

PROBLEM
You can see above, in the parser grammar (protocol.g) I can store all the children of the transition root as child1, child2, child3 and child4. Also, I have printed these. In the tree grammar, how can I store these and can perform operations upon these? Thank you

Answer 1

I'll instantiate java classes (will create java objects) e.g, The very first number in the tree will determine how many objects will be created, then, PAIR 5 0 will create an object with 2 arguments(5,0), PAIR 5 1 will create 2nd object with 2 arguments (5,1) and PAIR 5 2 will create 3rd object with 2 arguments (5,2).

Here is a simple way to create transitions and add pairs to them, and it requires only a small change to protocolWalker.g. First, though, here are the dummy Transitions and Pair classes that I'll use:

Transitions.java

import java.util.ArrayList;


public class Transitions {
    private ArrayList<Pair> pairs = new ArrayList<Pair>();

    public void addPair(Pair pair){
        System.out.println(String.format("Added pair %s to transitions", pair));
        pairs.add(pair);
    }

    @Override
    public String toString() {
        return "Pairs: " + pairs;
    }
}

Pair.java

public class Pair {
    private int a;
    private int b;

    public Pair(int a, int b){
        this.a = a;
        this.b = b;
    }

    @Override
    public String toString() {
        return String.format("(%d, %d)", a, b);
    }
}

Here is the modified protocolWalker.g.

protocolWalker.g (modified)

tree grammar protocolWalker;

options {
  language = Java;
  tokenVocab = protocol;   
  ASTLabelType = CommonTree;
}    


@header {
    package com.javadude.antlr3.x.tutorial;
    import java.util.List;
    import java.util.ArrayList;
}

@members { 
  //stores all the transitions objects as they get processed
  private ArrayList<Transitions> allTransitions = new ArrayList<Transitions>();

  //returns all the transitions
  public List<Transitions> getAllTransitions() { 
    return allTransitions;
  }
}


transitions
@init { 
        //create a Transitions object when the rule is hit
        Transitions transitions = new Transitions();

        //store it to be accessed later.
        allTransitions.add(transitions);
      } 
 : ^(TRANSITIONS INT transitions_pair[transitions]+) //pass the object to transitions_pair for each PAIR encountered
 {
     System.out.println("transitions=" + $INT.text);
 }
 ;

transitions_pair[Transitions transitions]
 : ^(PAIR a=INT b=INT) 
 {
     System.out.println("pair=" + $a.text + ", " + $b.text);
     //make a call to the Transitions object that was passed to this rule.
     transitions.addPair(new Pair($a.int, $b.int));
 }
 ;

(I renamed pair to transitions_pair because the rule is now tied to transitions-building.) The rule transitions calls transitions_pair, passing a new Transitions object at the same time. transitions_pair adds a new Pair object to the received Transitions object.

Rules in tree parsers and token parsers can be written to accept objects using the [ArgType argname,...] way. In this case, it makes it easier to visit the child PAIR trees.

I added a small change to Protocoltest.java to print out the stored transitions:

        ...
        //invoke the start symbol, rule parse
        walker.transitions();

        //get the stored transitions and print them out.            
        List<Transitions> transitions = walker.getAllTransitions();
        System.out.println(transitions);
        ...

Here is the new output for the walker:

pair=5, 0
Added pair (5, 0) to transitions
pair=5, 1
Added pair (5, 1) to transitions
pair=5, 2
Added pair (5, 2) to transitions
transitions=3
[Pairs: [(5, 0), (5, 1), (5, 2)]]

Here's a recap of the major changes I made:

• Added a way to store and return transitions from the walker.
• Added code to create a Transitions object in the rule transitions.
• Added code to pass the object to transitions_pair.
• Added code in the tester to retrieve the transitions from the walker and print them out.

I think you'll be all set once you implement your own Transitions class.