In pyparsing, how to I format a nested dictionary output?

Question

I'm using pyparsing to parse the following input:

%FSLAX45Y67*%

The output format I'm after, in dictionary form, is:

{
  'notation': 'absolute', 
  'zeros': 'leading', 
  'x': {
    'integer': 4, 
    'decimal': 5
  }, 
  'y': {
    'integer': 6, 
    'decimal': 7,
  }, 
  'gerber-command': 'FS'
}

The output I'm currently getting is:

{
  'notation': 'absolute', 
  'decimal': 6, 
  'zeros': 'leading', 
  'integer': 6, 
  'y': ([6, 6], {'integer': [(6, 0)], 'decimal': [(6, 1)]}), 
  'x': ([6, 6], {'integer': [(6, 0)], 'decimal': [(6, 1)]}), 
  'gerber-command': 'FS'
 }

(Note that my question is not about how to make the output look right, but rather how to make pyparsing arrange the data in the way I want.)

With the following code:

single_digit = pyp.Regex(r'(\d)').setParseAction(lambda t: int(t[0]))

cmd_format = pyp.Literal('FS')
cmd_format_opt_leading_zeros = pyp.Literal('L').setParseAction(pyp.replaceWith('leading'))
cmd_format_opt_trailing_zeros = pyp.Literal('T').setParseAction(pyp.replaceWith('trailing'))

format_zeros = ((cmd_format_opt_leading_zeros('zeros')) |
               (cmd_format_opt_trailing_zeros('zeros')))

format_notation = ((cmd_format_opt_absolute('notation')) |
                  (cmd_format_opt_incremental('notation')))

format_data = (single_digit)('integer') + single_digit('decimal')

gformat = (inst_del +
           cmd_format('gerber-command') +
           format_zeros +
           format_notation +
           'X' + (format_data)('x') + 
           'Y' + (format_data)('y') + 
           inst_end +
           inst_del)

(Some trivial definitions omitted). Any suggestions?

Answer 1

Use pyparsing Group to add structure to the returned tokens. This will probably do the job:

gformat = (inst_del +
           cmd_format('gerber-command') +
           format_zeros +
           format_notation +
           'X' + pyp.Group(format_data)('x') + 
           'Y' + pyp.Group(format_data)('y') + 
           inst_end +
           inst_del)

Pyparsing's default behavior is to just return a flat list of tokens, so as not to guess at structure just based on what terms are added in what order. For instance, if you had this:

aword = Word("A")
bword = Word("B")
cword = Word("C")

preface = aword + bword
body = cword
ending = aword + bword

totalExpr = preface + body + ending

print totalExpr.parseString("AA BB CCC A B").asList()

pyparsing would return just the list

['AA', 'BB', 'CCC', 'A', 'B']

If you want to apply structure (this is especially important to keep nested results names from stepping on each other, as you are seeing in the overlap of integer and decimal), use Group:

totalExpr = Group(preface) + body + Group(ending)

which gives:

[['AA', 'BB'], 'CCC', ['A', 'B']]

Here's how this would look if you added results names:

preface = aword("As") + bword("Bs")
body = cword
ending = aword("As") + bword("Bs")

totalExpr = Group(preface)("preface") + body("body") + Group(ending)("ending")
print totalExpr.parseString("AA BB CCC A B").dump()

Gives:

[['AA', 'BB'], 'CCC', ['A', 'B']]
- body: CCC
- ending: ['A', 'B']
  - As: A
  - Bs: B
- preface: ['AA', 'BB']
  - As: AA
  - Bs: BB

Because preface and ending are grouped, their duplicate names of "As" and "Bs" are kept separate.

Answer 2

You have to use setParseAction liberally to drop information you do not need. In format_data you would need a function which takes the output from single_digit and transforms it into the format you want.

def _format_data(x):
    return {"integer": x["integer"][0][0],
            "decimal": x["decimal"][0][0]}

format_data.setParseAction(_format_data)

Answer 3

interesting that why difficult questions always end up with recursion?

x=\
{
  'notation': 'absolute', 
  'zeros': 'leading', 
  'x': {
    'integer': 4, 
    'decimal': 5
  }, 
  'y': {
    'integer': 6, 
    'decimal': 7,
  }, 
  'gerber-command': 'FS'
}

def superPrint(inidic={},indent='  '):
    for k,v in inidic.items():
        if isinstance(v,dict):
            yield "\n%s'%s': {"%(indent,k)
            for i in superPrint(v,indent+' '*(len(k)+1)):
                yield i
            yield "\n%s},"%indent
        else:
            yield "\n%s'%s': '%s',"%(indent,k,v)

print '{%s\n}'%''.join(superPrint(x))

result:

{
  'y': {
    'integer': '6',
    'decimal': '7',
  },
  'x': {
    'integer': '4',
    'decimal': '5',
  },
  'zeros': 'leading',
  'notation': 'absolute',
  'gerber-command': 'FS',
}

NOTE,according to your description of quesion,I am not sure whether or not you want a "," in the last element of a dict.

Try deeper:

x=\
{
  'notation': 'absolute', 
  'zeros': 'leading', 
  'x': {
    'integer': 4, 
    'decimal': 5
  }, 
  'y': {
    'integer': 6, 
    'decimal': {'try':7,
                'tryHarder':{'wow':8,
                             'seemsGood':{'wooow':9}}},
  }, 
  'gerber-command': 'FS'
}

print '{%s\n}'%''.join(superPrint(x))

SEEMS good:

{
  'y': {
    'integer': '6',
    'decimal': {
            'try': '7',
            'tryHarder': {
                      'wow': '8',
                      'seemsGood': {
                                'wooow': '9',
                      },
            },
    },
  },
  'x': {
    'integer': '4',
    'decimal': '5',
  },
  'zeros': 'leading',
  'notation': 'absolute',
  'gerber-command': 'FS',
}