Parsekit: how to match individual quote characters?

Question

When using the parser Parsekit for the iPhone. Is it possible to include against a double quote? And things which are part of the special BNF? (Is it possible to escape sequences in a defined grammer?)

@start = doublequote+;
doublequote= '"'

Answer 1

Developer of ParseKit here.

By default you can match against quoted strings easily using the built-in QuotedString parser (which will match QuotedString tokens):

@start = quotes;
quotes = QuotedString+;

that would match input like: "foo" 'bar' "baz"

as three quoted strings: "foo", 'bar', "baz"

So this demonstrates that by default the ParseKit tokenizer (the PKTokenizer class) produces QuotedString tokens when encountering a " or '.

For more details on default tokenizer behavior, read the ParseKit tokenization documentation.

---

However, if you want quote chars (", ') to be recognized as standalone symbols rather than indicating the start or end of a quoted string, you must alter the tokenizer behavior first.

In code, you would alter tokenizer behavior by calling methods on your PKTokenizer object.

In grammars, you alter tokenizer behavior with tokenizer directives.

Tokenizer directives are special rules placed at the top of your grammar which start with a @ character. In this case, you want to change which characters are recognized as standalone symbol tokens by the tokenizer. Specifically, you want to add two chars as symbols with the @symbolState tokenizer directive.

You can do that in your grammar by changing it to:

@symbolState = '"' "'"; // a tokenizer directive stating ' and " should be recognized as standalone symbol tokens
                        // (by default they are start- and end-markers for quoted string tokens)

@start = stuff;
stuff = (Word | Symbol)+;

Given the same input as above, you would match separate quote symbols and words: ", foo, ", ', bar, ', ", baz, "