Can someone give an explanation on Lua's Interpreter and why is it so fast?

Question

I've been looking at the source code from Lua's standalone interpreter and so far I haven't been able to grok it.

Perhaps my Googling skills have worn out, but I haven't been able to find a good explanation as to how does it work. Lua's code seems optimized for performance, not readability.

Answer 1

There are two sources of perception of speed: script compilation and script execution.

Script compilation uses a recursive descent parser combined with a precedence expression parser. See http://www.lua.org/source/5.1/lparser.c.html.

Script execution interprets instructions for a virtual machine. This machine is fast because it is specialized to the semantics of Lua and also because it is a register-based virtual machine, which is faster than the stack-based virtual machine used in earlier versions of Lua. See http://www.lua.org/source/5.1/lopcodes.h.html.

Answer 2

As a broad overview, the Lua core operates by first compiling source text to an intermediate form called bytecode, which is then executed by an interpreter. Compilation to bytecode improves performance partly by performing the expensive lexical and syntax analysis once only, and partly by providing opportunities for optimizations that would be extremely difficult to perform without a compilation phase. The bytecode is the machine language of a register virtual machine. It has been designed to correspond well to the semantics of Lua, and to be easily interpreted.

The standalone executable simply loads the Lua core, acts on any options (such as -l and -e) that take effect ahead of a named script, compiles, loads, and executes the named script, and returns status to the OS in the usual way.

So that errors are handled consistently, one confusion in the lua.c is the delegation of all actual work aside from creation of the Lua state to the function pmain instead of just doing it in main. This allows pmain to be called in a protected context so that errors it throws will be caught rather than causing the abnormal termination of main.

While the source to the core of Lua is obscure, it is remarkably well documented. It does require a fair bit of background in compiler design, language design, virtual machines, and related topics to really understand it in depth.

There have been numerous academic papers about the language, its implementation, and its application. Those will provide a goldmine of information about why certain decisions were made, and in some cases, later revised.

The Lua User's Wiki has some pages devoted to exploration of the internal mechanisms of several features of the language, as well as lots of content about its use and extension.

Finally, I'll mention that the source code to the standard library modules is much more accessible that the source to the core. It will always provide solid examples of techniques for interfacing an external library to Lua.