The idea is to write your own Evaluator (that is subclass ACRFEvaluator). There are two methods you must write, evaluate
and test
. The evaluate
method can be copied code for another Evaluator in the source code, no big deal. It is the test
method where you add the lines to perform the task.
In general, the output of command line is split between stdout and the logger used. The results for each label (f-score and accuracy) end up in the logger output so that may be where we put the labels, otherwise perhaps a separate file for the labels, your choice. So here is a code example for a test method. I will output to stdout and you adjust to your tastes.
@Override
public void test(InstanceList gold, List returned, String description) {
for (Iterator it = evals.iterator(); it.hasNext(); ) {
ACRFEvaluator eval = (ACRFEvaluator) it.next();
eval.test(gold, returned, description);
int rows = returned.size();
for (int index = 0; index < rows; index++)
System.out.println("PREDICTED: " + returned.get(index).toString() + " vs. " + gold.get(index).toString());
}
}
}
For completeness, the class declaration is:
public class MyEvaluator extends ACRFEvaluator {
// body goes here
}
And the switch from the command line would look like
--eval "new MyEvaluator()"