How do I split an atom in Parslet?

Question

I'm building an SQL-like query language. I would like to be able to handle lists of items delimited by commas. I have successfully achieved this with this code:

class QueryParser < Parslet::Parser
  rule(:space) { match('\s').repeat(1) }
  rule(:space?) { space.maybe }

  rule(:delimiter) { space? >> str(',') >> space? }

  rule(:select) { str('SELECT') >> space? }
  rule(:select_value) { str('*') | match('[a-zA-Z]').repeat(1) }
  rule(:select_arguments) do
    space? >>
    (select_value >> (delimiter >> select_value).repeat).maybe.as(:select) >>
    space?
  end

  rule(:from) { str('FROM') >> space? }
  rule(:from_arguments) { match('[a-zA-Z]').repeat(1).as(:from) >> space? }

  rule(:query) { select >> select_arguments >> from >> from_arguments }
  root(:query)
end

Where something like SELECT id,name,fork FROM forks correctly outputs the {:select=>"id,name,fork"@7, :from=>"forks"@25} tree.

Now, instead of messing around with this later, I would like to be able to convert the SELECT arguments (id,name,fork in this case) into an Array. I can do this by running 'id,name,fork'.split ','. I cannot get the Parslet transformer to do this for me when applied. This my code for my query transformer:

class QueryTransformer < Parslet::Transform
  rule(select: simple(:args)) { args.split(',') }
end

When applied like so:

QueryTransformer.new.apply(
  QueryParser.new.parse('SELECT id,name,fork FROM forks')
)

The result is the same as when I didn't apply it: {:select=>"id,name,fork"@7, :from=>"forks"@25}.

The value I was hoping :select to be is an Array like this ["id","name","fork"].

My question is: how do I split the value of :select into an Array using transformers?

Answer 1

You need to put "as(:xxx)" on whatever part of the parse tree you want to be able to play with later.

Here I changed your rule(:select_value) to remember the values as a :value

rule(:select_value) { (str('*') | match('[a-zA-Z]').repeat(1)).as(:value) }

Now your parser outputs :

{:select=>[{:value=>"id"@7}, {:value=>"name"@10}, {:value=>"fork"@15}], :from=>"forks"@25}

Which is easy to transform using:

class QueryTransformer < Parslet::Transform
  rule(:value => simple(:val)) { val }
end

Then you get:

{:select=>["id"@7, "name"@10, "fork"@15], :from=>"forks"@25}

So in full the code is as follows :-

require 'parslet'


class QueryParser < Parslet::Parser
  rule(:space) { match('\s').repeat(1) }
  rule(:space?) { space.maybe }

  rule(:delimiter) { space? >> str(',') >> space? }

  rule(:select) { str('SELECT') >> space? }

  rule(:select_value) { (str('*') | match('[a-zA-Z]').repeat(1)).as(:value) }

  rule(:select_arguments) do
    space? >>
    (select_value >> (delimiter >> select_value).repeat).maybe.as(:select) >>
    space?
  end

  rule(:from) { str('FROM') >> space? }
  rule(:from_arguments) { match('[a-zA-Z]').repeat(1).as(:from) >> space? }

  rule(:query) { select >> select_arguments >> from >> from_arguments }
  root(:query)
end

puts QueryParser.new.parse('SELECT id,name,fork FROM forks') 
# =>  {:select=>[{:value=>"id"@7}, {:value=>"name"@10}, {:value=>"fork"@15}], :from=>"forks"@25}

class QueryTransformer < Parslet::Transform
  rule(:value => simple(:val)) { val }
end

puts QueryTransformer.new.apply(
  QueryParser.new.parse('SELECT id,name,fork FROM forks')
)
# => {:select=>["id"@7, "name"@10, "fork"@15], :from=>"forks"@25}