JParsec bug in parsing simple String expressions

Question

I need a parser for my java application, so I discovered the JParsec library and went through a tutorial on their website with a simple calculator grammar. It seems that I should be able to do what I planned using the library, hence started prototyping changing parsing decimal literals from the sample Calculator to parsing very simple Identifiers as Strings. The only operation I want to work right now is '+', to concatenate strings, but I keep getting error messages. Here's my code:

import org.codehaus.jparsec.OperatorTable;
import org.codehaus.jparsec.Parser;
import org.codehaus.jparsec.Parsers;
import org.codehaus.jparsec.Scanners;
import org.codehaus.jparsec.Terminals;
import org.codehaus.jparsec.Tokens.Fragment;
import org.codehaus.jparsec.functors.Binary;
import org.codehaus.jparsec.functors.Map;

public class Test

{
enum BinaryOperator implements Binary<String> {
    PLUS {
        public String map(String a, String b) {
            return a + b;
        }
    }
}

static final Parser<Void> IGNORED = Scanners.WHITESPACES.skipMany();

static final Parser<String> TEXT = Terminals.Identifier.TOKENIZER.map(new Map<Fragment, String>() {
    public String map(Fragment arg0) {
        return arg0.text()+"NEWTOSEEIFITSWORKING";
    }
});

private static final Terminals OPERATORS = Terminals.operators("+");

static final Parser<?> TOKENIZER = Parsers.or(Terminals.Identifier.TOKENIZER, OPERATORS.tokenizer());

static Parser<?> term(String... names) {
    return OPERATORS.token(names);
}

static <T> Parser<T> op(String name, T value) {
    return term(name).retn(value);
}

static Parser<String> textparser(Parser<String> atom) { 
    Parser<String> parser = new OperatorTable<String>().infixl(op("+", BinaryOperator.PLUS), 10).build(atom);
    return parser;
}

public static final Parser<String> TEXTPARSER = textparser(TEXT).from(TOKENIZER, IGNORED);

public static void main(String[] args)
{
    System.out.println(TEXTPARSER.parse("hello+world"));
}}

And here are errors I'm getting:

Exception in thread "main" org.codehaus.jparsec.error.ParserException: Cannot scan characters on tokens.
line 1, column 1
    at org.codehaus.jparsec.Parser.asParserException(Parser.java:591)
    at org.codehaus.jparsec.Parser.run(Parser.java:600)
    at org.codehaus.jparsec.MapParser.apply(MapParser.java:30)
    at org.codehaus.jparsec.Parser.run(Parser.java:597)
    at org.codehaus.jparsec.MapParser.apply(MapParser.java:30)
    at org.codehaus.jparsec.Parser.run(Parser.java:597)
    at org.codehaus.jparsec.BindNextParser.apply(BindNextParser.java:37)
    at org.codehaus.jparsec.Parser.run(Parser.java:597)
    at org.codehaus.jparsec.Sequence2Parser.apply(Sequence2Parser.java:32)
    at org.codehaus.jparsec.Parser.run(Parser.java:597)
    at org.codehaus.jparsec.ParserInternals.runNestedParser(ParserInternals.java:116)
    at org.codehaus.jparsec.NestedParser.apply(NestedParser.java:32)
    at org.codehaus.jparsec.Parser.run(Parser.java:597)
    at org.codehaus.jparsec.Sequence2Parser.apply(Sequence2Parser.java:32)
    at org.codehaus.jparsec.Parser.run(Parser.java:597)
    at org.codehaus.jparsec.Parsers.parse(Parsers.java:97)
    at org.codehaus.jparsec.Parser.parse(Parser.java:535)
    at org.codehaus.jparsec.Parser.parse(Parser.java:546)
    at org.codehaus.jparsec.Parser.parse(Parser.java:551)
    at Test.main(Test.java:52)
Caused by: java.lang.IllegalStateException: Cannot scan characters on tokens.
    at org.codehaus.jparsec.ParserState.characters(ParserState.java:56)
    at org.codehaus.jparsec.PatternScanner.apply(PatternScanner.java:36)
    at org.codehaus.jparsec.ReturnSourceParser.apply(ReturnSourceParser.java:32)
    at org.codehaus.jparsec.Parser.run(Parser.java:597)
    ... 18 more

Any idea what I'm doing wrong? Any help is greatly appreciated!

Answer 1

Just stumble on this old question browsing the web. I think your problem is you are using a Parser to parse a String object. Parser combinators are strongly typed on the type of objects they produce, and specifically the combinators on Terminals generate tokens, not strings.