XML grammar for OkudaKit

Question

I'm trying to write an XML grammar for OkudaKit. I noticed that the bundled HTML grammar works for simple XML, but fails with namespaced elements or attributes, so the first thing I want to do is add support for these. Using the HTML grammar as a template, I defined ns and then added it to tagName and attrName:

@multiLineComments = '<!--' '-->';
@commentState = '<';
@commentState.fallbackState = delimitState;

@delimitedString = '<?' '?>' nil;
@delimitedString = '<!DOCTYPE' '>' nil;
@delimitState.fallbackState = symbolState;

@start          = any*;
any             = element | text | doctype | pi | comment;

pi              = DelimitedString('<?', '?>');

doctype         = DelimitedString('<!DOCTYPE', '>');
element         = emptyTag | startTag elementContent* endTag;
elementContent  = element | text | comment | pi;
text            = /[^<]+/;

emptyTag        = lt tagName attr* fwdSlash gt;
startTag        = lt junk? tagName attr* gt;
endTag          = lt fwdSlash tagName gt;

ns              = Word colon;

tagName         = ns? Word;

attr            = attrName eq attrValue;
attrName        = ns? Word;
attrValue       = QuotedString;

eq              = '=';
lt              = '<';
gt              = '>';
fwdSlash        = '/';
colon           = ':';

comment         = Comment;

Seems like it should work (if I understand the "?" properly, which I probably don't) but the output is messed up. Below is my test document:

<?xml version="1.0" encoding="utf-8"?>
<test cats:dogs="television">
    <peas vegetable="box" >
        <orange />
        <!-- the following makes no sense -->
        <blue lion:mouse="cold"/>
        <red car:desk="apple">
            < envelope></ envelope>
            <![CDATA[lorem ipsum]]>
            <dwarves>
                <dwarf>Sleepy</dwarf>
                <dwarf>Dopey</dwarf>
            </dwarves>
        </red>
    </peas>
</test>

And this is what it looks like after my broken grammar mangles it (ignore the formatting, that's NSXMLDocument's pretty printing):

<?xml version="1.0" encoding="utf-8"?>
< :cats=dogstest"television">
    < =vegetable"box"peas>
        <>orange</>orange
        <!-- the following makes no sense -->
        < :lion=mouseblue"cold"></>blue
        < :car=deskred"apple">&lt; envelope>&lt;/ envelope> lorem ipsum<>dwarves
                <>dwarfSleepy</>dwarf
                <>dwarfDopey</>dwarf
            </>dwarves
        </>red
    </>peas
</>test

I'm interpreting the "?" to mean "optional" but I think that's wrong. I didn't find it covered in the ParseKit grammar guide.

Answer 1

Developer of OkudaKit/ParseKit here. I've fixed the root problem which was causing the issue you ran into.

My OkudaKit HTML grammar didn't support CDATA or QNames. I have enhanced the HTML Grammar in the OkudaKit SVN repo (in trunk). Please update your working copy and you'll see the fix. The HTML grammar should now serve your needs (let me know if you run into issues).

Two Final things/tricks to keep in mind (which I had forgotten myself):

1. The Grammar Productions for which you define CSS Rules MUST be Terminal Grammar Productions.

   Here's some examples of Teminal Productions (hint: they don't point to other non-terminal productions):

   colon = ':';
   prefix = Word;
   comment = Comment;
   

   These are not Terminal Productions:

   qName = qualifiedName | unqualifiedName;
   @start = any*;
   

2. ALL Terminal Productions in your Grammar MUST have CSS Rules defined in the CSS file.

The reason your first attempt at a fix didn't work is that you were not respecting the two rules above. When you don't follow those two rules, the output gets borked/reordered in strange ways.

Very sorry, I know this information is not included anywhere with OkudaKit, so there's no way you could have known. I will try to fix that in the future.