SBT/Play2 multi-project setup does not include dependant projects in classpath in run/test

Question

I have following SBT/Play2 multi-project setup:

import sbt._
import Keys._
import PlayProject._

object ApplicationBuild extends Build {
  val appName         = "traveltime-api"
  val appVersion      = "1.0"

  val appDependencies = Seq(
    // Google geocoding library
    "com.google.code.geocoder-java" % "geocoder-java" % "0.9",
    // Emailer
    "org.apache.commons" % "commons-email" % "1.2",
    // CSV generator
    "net.sf.opencsv" % "opencsv" % "2.0",

    "org.scalatest" %% "scalatest" % "1.7.2" % "test",
    "org.scalacheck" %% "scalacheck" % "1.10.0" % "test",
    "org.mockito" % "mockito-core" % "1.9.0" % "test"
  )

  val lib = RootProject(file("../lib"))
  val chiShape = RootProject(file("../chi-shape"))

  lazy val main = PlayProject(
    appName, appVersion, appDependencies, mainLang = SCALA
  ).settings(
    // Add your own project settings here
    resolvers ++= Seq(
      "Sonatype Snapshots" at
        "http://oss.sonatype.org/content/repositories/snapshots",
      "Sonatype Releases" at
        "http://oss.sonatype.org/content/repositories/releases"
    ),
    // Scalatest compatibility
    testOptions in Test := Nil
  ).aggregate(lib, chiShape).dependsOn(lib, chiShape)
}

As you can see this project depends on two independant subprojects: lib and chiShape.

Now compile works fine - all sources are correctly compiled. However if I try run or test, neither task in runtime has classes from subprojects on classpath loaded and things go haywire with NoClassFound exceptions.

For example - my application has to load serialized data from file and it goes like this: test starts FakeApplication, it tries to load data and boom:

[info] CsvGeneratorsTest:
[info] #markerFilterCsv 
[info] - should fail on bad json *** FAILED ***
[info]   java.lang.ClassNotFoundException: com.library.Node
[info]   at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
[info]   at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
[info]   at java.security.AccessController.doPrivileged(Native Method)
[info]   at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
[info]   at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
[info]   at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
[info]   at java.lang.Class.forName0(Native Method)
[info]   at java.lang.Class.forName(Class.java:264)
[info]   at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:622)
[info]   at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1593)
[info]   ...

Strangely enough stage creates a directory structure with chi-shapes_2.9.1-1.0.jar and lib_2.9.1-1.0.jar in staged/.

How can I get my runtime/test configurations get subprojects into classpath?

Update:

I've added following code to Global#onStart:

  override def onStart(app: Application) {
    println(app)
    ClassLoader.getSystemClassLoader.asInstanceOf[URLClassLoader].getURLs.
      foreach(println)
    throw new RuntimeException("foo!")
  }

When I launch tests, the classpath is very very ill populated, to say at least :)

FakeApplication(.,sbt.classpath.ClasspathUtilities$$anon$1@182253a,List(),List(),Map(application.load-data -> test, mailer.smtp.test-mode -> true))
file:/home/arturas/Software/sdks/play-2.0.3/framework/sbt/sbt-launch.jar
[info] CsvGeneratorsTest:

When launching staged app, there's a lot of stuff, how it's supposed to be :)

$ target/start
Play server process ID is 29045
play.api.Application@1c2862b
file:/home/arturas/work/traveltime-api/api/target/staged/jul-to-slf4j.jar

That's strange, because there should be at least testing jars in the classpath I suppose?

Answer 1

It seems I've solved it.

The culprit was that ObjectInputStream ignores thread local class loaders by default and only uses system class loader.

So I changed from:

  def unserialize[T](file: File): T = {
    val in = new ObjectInputStream(new FileInputStream(file))
    try {
      in.readObject().asInstanceOf[T]
    }
    finally {
      in.close
    }
  }

To:

  /**
   * Object input stream which respects thread local class loader.
   *
   * TL class loader is used by SBT to avoid polluting system class loader when
   * running different tasks.
   */
  class TLObjectInputStream(in: InputStream) extends ObjectInputStream(in) {
    override protected def resolveClass(desc: ObjectStreamClass): Class[_] = {
      Option(Thread.currentThread().getContextClassLoader).map { cl =>
        try { return cl.loadClass(desc.getName)}
        catch { case (e: java.lang.ClassNotFoundException) => () }
      }
      super.resolveClass(desc)
    }
  }

  def unserialize[T](file: File): T = {
    val in = new TLObjectInputStream(new FileInputStream(file))
    try {
      in.readObject().asInstanceOf[T]
    }
    finally {
      in.close
    }
  }

And my class not found problems went away!

Thanks to How to put custom ClassLoader to use? and http://tech-tauk.blogspot.com/2010/05/thread-context-classlaoder-in.html on useful insight about deserializing and thread local class loaders.

Answer 2

This sounds similar to this bug https://play.lighthouseapp.com/projects/82401/tickets/659-play-dist-broken-with-sub-projects, though that bug is about dist and not test. I think that the fix has not made it to the latest stable release, so try building Play from source (and don't forget to use aggregate and dependsOn as demonstrated in that link.

Alternatively, as a workaround, inside sbt, you can navigate to the sub-project with project lib and then type test. It's a bit manual, but you can script that if you'd like.