Building a string parser for user command and control?

Question

My goal is to build a command parser that has basic syntax and multiple possible branches at each point. These commands come from users of the system and are text input (no GUI). The basic syntax is base_command [sub_command [optional_parameters]], where optional_parameters is a space seperated list of parameters.

Examples:

add event "Program-a-thon"
add notification program-a-thon meeting
add pattern "follow the white rabbit"
remove event "Program-a-thon"
remove notification meeting
remove pattern "follow the white rabbit"
edit pattern "follow the white rabbit" "follow the white tiger"
reload settings

As you can see, I have 4 base commands (add, remove, edit and reload). However, I anticipate adding more in the future. I also have 4 subcommands (event, notification, pattern and settings). However, edit only has 1 subcommand and settings only applies to the reload base command. Not all subcommands are associated with all base commands. Finally, I have parameters. For simplicity, I am planning on making these position based.

I am writing this application in Python (utilizing the 2.7 branch).

So far, I've come up with this:

• Utilize the .split() command and take the first two results to get the command and base command. The third index (if it exists) will be the positional arguments.

  OR

• Utilize a regular expression to get the base command and sub-command. The third index will be positional arguments.

Then utilizing this information, look at a dictionary like this to see if the command/sub-command option is viable. If it is, call the method described in the commands[base_command][sub_command]['method'] key to perform my action.

The upside of this is that I can easily tell if the command/subcommand combination is valid. The downside is maintaining such a dictionary and possibly extending it if I wish to add more options or information. For just these few commands I already have a dictionary like this (with some supporting attributes)

commands = 
{
    'add' : 
    {
        'help_text' : "Add a `event`, `notification`, or `pattern`",
        'sub_commands': {
                'event' : {
                    'help_text' : "Add an event",
                    'method' : "add_event",
                    'permissions' : 'trusted',
                },
                'notification' : {
                    'help_text' : "Recieve a notification for events that are tagged with the values you wish (tag1 [,tag2[,tag3[,tag4]]])",
                    'method' : "add_notification",
                    'permissions' : 'all',
                },
                'pattern' : {
                    'help_text' : "Add a pattern you wish the system to watch for in new event descriptions",
                    'method' : "add_pattern",
                    'permissions' : 'all',
                },
            },
        'sub_command_required' : True,
        'method' : None,
    },
    'remove':
    {
        'help_text': "Remove a `event`, `notification`, or `pattern`",
        'sub_commands': {
                'event' : {
                    'help_text' : "Remove an event",
                    'method' : "remove_event",
                    'permissions' : 'trusted',
                },
                'notification' : {
                    'help_text' : "Remove a notification for specific tags (tag1 [,tag2[,tag3[,tag4]]])",
                    'method' : "remove_notification",
                    'permissions' : 'all',
                },
                'pattern' : {
                    'help_text' : "Add a pattern you wish the system to watch for in new event descriptions",
                    'method' : "remove_pattern",
                    'permissions' : 'all',
                },
            },
        'sub_command_required' : True,
        'method' : None,            
    },
    'edit':
    {
        'help_text': "Edit an existing `pattern`",
        'sub_commands': {
                'pattern' : {
                    'help_text' : "Edit a pattern pattern to watch for in new event descriptions (old_pattern, new_pattern)",
                    'method' : "edit_pattern",
                    'permissions' : 'all',
                },
            },
        'sub_command_required' : True,
        'method' : None,
    },
    'reload':
    {
        'help_text': "Reload application options",
        'sub_commands': {
                'settings' : {
                    'help_text' : "Reload the settings of the application",
                    'method' : "reload_settings",
                    'permissions' : 'owner',
                },
            },
        'sub_command_required' : True,
        'method' : None,
    },
}

My questions are, how do I properly design the parser so that I can

1. Extend the command tree easily in the future?
2. Have a combination of sub-commands that may or may not apply to all base commands?

And

3. Is utilizing a dictionary, like above, worth the hassle of maintenance when adding commands? I realize that commands will be 'relatively' stable (once added, modifications will be rare to the dictionary itself). However, when I do add them, they will probably come in at least pairs (add and remove), meaning the potential for messing up the dictionary occurs in multiple places.

Answer 1

Have a look at the Python argparse and cmd modules. If you'd like to make an interactive UI, subclass cmd.Cmd and for each of the commands, you can use argparse to parse the subcommand and options.

import readline # for history and autocomplete
import cmd

class MyShell(cmd.Cmd):
    def do_add(self, line):
        "adds various things"
        print "add called with arguments", line # here you can use argparse on line


>>> MyShell().cmdloop()
(Cmd) xyz
*** Unknown syntax: xyz
(Cmd) help

Documented commands (type help <topic>):
========================================
add  help

(Cmd) help add
adds various things
(Cmd) add
add called with arguments
(Cmd) add 1 2 3 4
add called with arguments 1 2 3 4
(Cmd)

Answer 2

Parsing this kind of commands is very much like parsing command line options and the fact that you have one word commands and arguments just makes it easier.

What you basically can do is to read a line and split it using a regex. The reason I would choose for a regex is that it makes it easier to split the string and keep the text in quotes together.

When the line is split you have to lookup the command, or you can lookup the subcommand first. Since an 'event' can be added and removed, it could be logical to combine them in the dictionary under 'event' instead of 'add' and 'remove', because you do the add and remove operation on an event.

The values in the dictionary could be command classes that have the knowledge to parse the arguments, run the command and print the help text. This way the main dictionary will stay readable and maintainable and the commands could be tested individually.

A small example in python like pseudo code of what I mean:

class commandBase
    def execute(subcommand, args):
        self.subCommands[subcommand].method(args);

class eventCommand : commandBase
    def __init__(self):
        self.subCommands = {
           "add": { "help_text" : "add an event", method: self.add }
           "remove": { "help_text" : "remove an event", method: self.remove }
        }

command = { "event" : new eventCommand }
parsedLine = parseLine(line)
command[parsedLine[1]].execute(parsedLine[0], pardesLine[2])