pyparsing performance and memory usage

Question

Pyparsing worked fine for a very small grammar, but as the grammar has grown, the performance went down and the memory usage through the roof.

My current gramar is:

newline = LineEnd ()
minus = Literal ('-')
plus = Literal ('+')
star = Literal ('*')
dash = Literal ('/')
dashdash = Literal ('//')
percent = Literal ('%')
starstar = Literal ('**')
lparen = Literal ('(')
rparen = Literal (')')
dot = Literal ('.')
comma = Literal (',')
eq = Literal ('=')
eqeq = Literal ('==')
lt = Literal ('<')
gt = Literal ('>')
le = Literal ('<=')
ge = Literal ('>=')
not_ = Keyword ('not')
and_ = Keyword ('and')
or_ = Keyword ('or')
ident = Word (alphas)
integer = Word (nums)

expr = Forward ()
parenthized = Group (lparen + expr + rparen)
trailer = (dot + ident)
atom = ident | integer | parenthized
factor = Forward ()
power = atom + ZeroOrMore (trailer) + Optional (starstar + factor)
factor << (ZeroOrMore (minus | plus) + power)
term = ZeroOrMore (factor + (star | dashdash | dash | percent) ) + factor
arith = ZeroOrMore (term + (minus | plus) ) + term
comp = ZeroOrMore (arith + (eqeq | le | ge | lt | gt) ) + arith
boolNot = ZeroOrMore (not_) + comp
boolAnd = ZeroOrMore (boolNot + and_) + boolNot
boolOr = ZeroOrMore (boolAnd + or_) + boolAnd
match = ZeroOrMore (ident + eq) + boolOr
expr << match
statement = expr + newline
program = OneOrMore (statement)

When I parse the following

print (program.parseString ('3*(1+2*3*(4+5))\n') )

It takes quite long:

~/Desktop/m2/pyp$ time python3 slow.py 
['3', '*', ['(', '1', '+', '2', '*', '3', '*', ['(', '4', '+', '5', ')'], ')']]

real    0m27.280s
user    0m25.844s
sys 0m1.364s

And the memory usage goes up to 1.7 GiB (sic!).

Have I made some serious mistake implementing this grammar or how else can I keep memory usage in bearable margins?

Answer 1

After importing pyparsing enable packrat parsing to memoize parse behavior:

ParserElement.enablePackrat()

This should make a big improvement in performance.