Vim syntax match first and second elements recursively

Question

I'm trying to write a set of syntax rules for matching a nestable syntax. Basically, at the simplest level, everything is either a token or a bracket. Here's an example of such a syntax:

[
    [a b c]
    [1 2 3]
    [foo
        [bar baz]]
]

I want the first element of each "list" to be colored, so in this case, a, 1, foo, and bar. I also want the second element of each list to be colored, but in a different way. In this case, b, 2, and baz. I don't care about anything else. There are no delimiters between items besides whitespace.

How can I do this with the vim pattern system? I've managed to get a simple system working, but it handles nested lists poorly. The trouble I'm having is reliably matching the first and second items of each list separately, without screwing with the other ones.

Any help?

Answer 1

The following set of rules highlights your syntax:

highlight       FirstToken      ctermfg=green       guifg=green
highlight       SecondToken     ctermfg=red         guifg=red

syntax region   FirstTuple      start="\(\[\_s*\)\@<=\["    end="]"
            \ containedin=FirstTuple,OtherTuple
            \ nextgroup=SecondToken
            \ skipwhite skipnl skipempty
syntax region   OtherTuple      start="\(\[\_s*\)\@<!\["    end="]"
            \ containedin=FirstTuple,OtherTuple

syntax match    FirstToken      "\(\[\_s*\)\@<=[^ [\]]\+"
            \ containedin=FirstTuple,OtherTuple
            \ nextgroup=SecondToken
            \ skipwhite skipnl skipempty
syntax match    SecondToken     "[^ [\]]\+"
            \ contained

The FirstTuple region matches tuples, of either tokens or of other tuples, that are the first in their parent tuple; OtherTuple is used for all other tuples. This is done via mutually exclusive lookbehinds, and proper nesting is handled automatically. FirstToken uses a positive lookbehind to only match at the beginning of a region. FirstToken and FirstTuple use the nextgroup attribute to attempt to match SecondToken immediately after themselves, ignoring whitespace, newlines, and empty lines. SecondToken uses the contained attribute to avoid matching everywhere by itself.