Scala: parsing multiple files using Scala's combinators

Question

I am writing a DSL using Scala parser combinators and have a working version that can read a single file and parse it. However, I would like to split my input into several files where some files are 'standard' and can be used with any top-level file. What I would like is something like:

import "a.dsl"
import "b.dsl"
// rest of file using {a, b}

It isn't important what order the files are read in or that something is necessarily 'defined' before being referred to so parsing the top-level file first then parsing the closure of all imports into a single model is sufficient. I will then post-process the resulting model for my own purposes.

The question I have is, is there a reasonable way of accomplishing this? If necessary I could iterate over the closure, parsing each file into a separate model, and manually 'merge' the resulting models but this feels clunky and seems ugly to me.

BTW, I am using an extension of StandardTokenParsers, if that matters.

Answer 1

I think the only approach would be to open and parse the file indicated by the import directly. From there you can create a sub-expression tree for the module. You may not need to manually merge the trees when parsing, for example if your already using ^^ and/or ^^^ to return your own Expressions then you should be able to simply emit a relevant expression type in the correct place within the tree, for example:

import scala.util.parsing.combinator.syntactical.StandardTokenParsers
import scala.io.Source

object Example {

  sealed trait Expr

  case class Imports(modules: List[Module]) extends Expr
  case class Module(modulePath: String, root: Option[Expr]) extends Expr
  case class BracedExpr(x: String, y: String) extends Expr
  case class Main(imports: Imports, braced: BracedExpr) extends Expr


  class BlahTest extends StandardTokenParsers {

    def importExpr: Parser[Module] = "import" ~> "\""  ~> stringLit <~ "\"" ^^ {
      case modulePath =>

        //you could use something other than `expr` below if you
        //wanted to limit the expressions available in modules
        //e.g. you could stop one module importing another.
        phrase(expr)(new lexical.Scanner(Source.fromFile(modulePath).mkString)) match {
          case Success(result, _) =>
            Module(modulePath, Some(result))

          case failure : NoSuccess =>
            //TODO log or act on failure
            Module(modulePath, None)
        }
    }

    def prologExprs = rep(importExpr) ^^ {
      case modules =>
        Imports(modules)
    }

    def bracedExpr = "{" ~> stringLit ~ "," ~ stringLit <~ "}" ^^ {
      case x ~ "," ~ y =>
        BracedExpr(x, y)
    }

    def bodyExprs = bracedExpr

    def expr = prologExprs ~ bodyExprs ^^ {
      case prolog ~ body =>
        Main(prolog, body)
    }

  }

}

You could simply add an eval to your Expression trait, implement each eval as necessary on the sub-classes and then have a visitor recursively descend your AST. In this manner you would not need to manually merge expression trees together.