Tokenizing Rule Precedence Pyparsing

Question

Below is my toy parser:

from pyparsing import Word,OneOrMore,alphanums,nums

number=Word(nums)
word=OneOrMore(Word(alphanums))
greeting = word+(number)

tests = "Hello3 World 1234"
print tests, "->", greeting.parseString(tests)

The results I was expecting to get were

['Hello3','World','1234']

but instead I get the error:

ParseException: Expected W:(0123...) (at char 17), (line:1, col:18)

Should not the '1234' string not be matched by the 'number' rule first, and so subsequently be ignored by the 'word' rule?

I suspect it is late at night and I have missed something utterly elementary in the documentation, but I have looked through the 'getting started with pyparsing' O'Reilly pdf twice, and can't find a solution. Thanks for any help.

Answer 1

This is matching your "1234" against the word rule since you said it could be alpha or numeric. The word rules keeps capturing the terms including "1234" and so the error is that the number rule is never hit.

If you're rule for word is closer to python's definition of an identifier (can't start with a digit), this will fix your parser:

from pyparsing import Word, OneOrMore, alphas, alphanums, nums

number=Word(nums)
word=OneOrMore(Word(initChars=alphas, bodyChars=alphanums))
greeting = word+(number)

tests = "Hello3 World 1234"
print tests, "->", greeting.parseString(tests)