I need a tool to parse Lua tables, preferrably in Ruby or Java

Question

I need a tool to parse Lua table expressions. If all else fails, I will eventually just code a small Lua module to convert tables to XML, but for the time being, I am interested in a Ruby library doing that, but failing that, I would accept tool in any language, provided I can look at its source.

Here is an example snippet (it's a WoW addon output):

CT_RaidTracker_RaidLog = {
{
    ["PlayerInfos"] = {
        ["Nyim"] = {
            ["race"] = "Orc",
            ["guild"] = "Excubitores Noctae",
            ["sex"] = 2,
            ["class"] = "HUNTER",
            ["level"] = 70,
        },
        ["Zyrn"] = {
            ["race"] = "BloodElf",
            ["guild"] = "Excubitores Noctae",
            ["sex"] = 2,
            ["class"] = "WARLOCK",
            ["level"] = 70,
        },
...

Basic idea is, nested associative arrays. Any help or pointer will be examined, any idea is appreciated.

EDIT #1

Due to the disputes, let me clarify what did I try. I complemented the string/regex replacing chain provided by one of the participants, like so:

str.gsub(/--.+$/, "").gsub("=", ":").gsub(/[\[\]]/,"").gsub('" :','":').gsub(/,\s*\n(\s*)}/, "\n\\1}")

I (1) added removal of Lua comments, (2) replaced one of the regex replacers: when you have the last element in an object/array, it still has a comma after it, so that must be covered and the comma properly removed.

Do you notice the double opening curly braces? JSON doesn't like having anonymous objects. It looks like that:

"xxx" = {
  {
    ["aaa"} = {
      ["bbb"] = {
        "ccc" = 7
        "ddd" = "a string"
        "eee" = "a date/time pattern"
      }
    },
    ["qqq"} = {
      "hm" = "something"
    }
  },
  {
    ["aaa"] = {
    -- ...
    },
    ["qqq"] = {
    -- ...
    }
  }
}

Basically on the root level, we actually have a list/array of similar objects, both having "aaa" and "qqq" section, to follow the example. However, in Lua that is obviously allowed, while in JSON it isn't. Because the opening curly braces are treated like "start an object" but that object doesn't have a name.

I tried to detect that case with regex and replace curly braces with "[]" pairs. While the resulting regex worked, the problem was the same: OK, we define an array of similar objects instead, but the declaration of the array is still nameless.

A possible solution would be instead of detecting and replacing those braces with [], to christen the objects with indexes, like: "0" = { "aaa" = {...} }, "1" = { "aaa" = {... } }, etc. That (hopefully final) workaround will probably make it work... Will report back again. ;)

Answer 1

Skipping the first line and then some ad hoc transformation to JSON.

s=File.readlines("test.luatable")[1..-1].join
JSON.parse(s.gsub("=", ":").gsub(/[\[\]]/,"").gsub('" :','":').gsub(/,\n(.+)\}/,"\n\\1}"))
=> {"PlayerInfos"=>{"Nyim"=>{"guild"=>"Excubitores Noctae", "class"=>"HUNTER",  
    "level"=>70, "sex"=>2, "race"=>"Orc"}, "Zyrn"=>{"guild"=>"Excubitores Noctae", 
    "class"=>"WARLOCK", "level"=>70, "sex"=>2, "race"=>"BloodElf"}}}

Answer 2

I'm probably stating the obvious, but Lua can certainly parse Lua tables. And you can "embed" Lua in pretty much any mainstream language including Java and Ruby (scroll down the link for Java and Ruby bindings). By embed, I mean parsing source files, calling Lua functions and exploring tables, may be even calling functions written in your host language from Lua. It's possible that these binding libraries are more work than exporting your tables to XML/JSON, but it's worth looking at them at least

Edit: level 70? That's so last decade ;)

Answer 3

It's simple to write a Lua program that outputs tables in XML but it depends on how you want the XML formatted. See also LuaXML, which has xml.save (but is written in C) and this question.

Answer 4

It is probably going to be simpler to use JSON than xml on this case.

The translation from lua tables is nearly 1-to-1 (change = to :, and remove [ and ] from the keys). This is the JSON equivalent of your example:

{
  "PlayerInfos": {
    "Nyim": {
      "race": "Orc",
      "guild": "Excubitores Noctae",
      "sex": 2,
      "class": "HUNTER",
      "level": 70
    },
    "Zyrn": {
      "race": "BloodElf",
      "guild": "Excubitores Noctae",
      "sex": 2,
      "class": "WARLOCK",
      "level": 70
    },

...

Besides, Rails has built-in JSON-parsing (via JSON::parse).

In order to read it from a ruby app, you would have to do something similar to this:

require 'json' # This is already included on Rails apps

info = JSON::parse(File.read("PlayerInfos.json"))

Then the player infos would be available at:

player_infos = info["PlayerInfos"]

There's also a java JSON parser, but I have no experience with it.

Answer 5

Try this code

function toxml(t,n)
        local s=string.rep(" ",n)
        for k,v in pairs(t) do
                print(s.."<"..k..">")
                if type(v)=="table" then
                        toxml(v,n+1)
                else
                        print(s.." "..v)
                end
                print(s.."</"..k..">")
        end
end

toxml(CT_RaidTracker_RaidLog,0)

Answer 6

You mention you can only use Java, Ruby or PHP for the parsing of this. An option is to use a tool like ANTLR to generate a little parser for you.

The ANTLR grammar:

grammar Test;

parse
  :  Identifier '=' table EOF
  ;

table
  :  '{' (entry (',' entry)* ','?)? '}'
  ;

entry
  :  key ('=' value)?
  |  String
  |  Number
  ;

key
  :  '[' (String | Number) ']'
  |  Identifier
  ;

value
  :  String 
  |  Number
  |  Identifier
  |  table
  ;

Identifier
  :  ('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z' | '_' | '0'..'9')*
  ;

String
  :  '"' ~'"'* '"'
  ;

Number
  :  '0'..'9'+
  ;

Space
  :  (' ' | '\t' | '\r' | '\n') {skip();}
  ;

generates a parser that can take input like:

Table = {
  ["k1"] = "v1",
  ["k2"] = {["x"]=1, ["y"]=2},
  ["k3"] = "v3"
}

and transform it into:

alt text http://img59.imageshack.us/img59/7112/treef.png

(click here for a full resolution of the image)

Writing an XML from that tree structure is child's play.

But like I said, Lua tables can look quite different from the grammar I posted above: strings can look like:

'a string'
[===[ also ]==] a string ]===]

and keys and values can consist of expressions. But if the trees always look like how you posted it, it might be an option for you.

Good luck!

Answer 7

I needed to do this in Java, so I wrote a small library:

https://github.com/mutantbob/jluadata

Answer 8

Untested code. And, I'm just following Sbk's link here, so I don't deserve any credit at all.

require 'rubyluabridge'

def lua_data_to_ruby_hash(data)
    luastate = Lua::State.new
    luastate.eval "d = #{data}"
    return luastate.d.to_hash
end

Answer 9

Can I point out that Lua does not have regex capabilities, just pattern matched text replacement.