Scala Compiler doesn't terminate (programmatically invoked)

Question

I am programmatically compiling Scala code with this piece of code:

val compiler = new Global(settings, reporter)
val run = new compiler.Run
run compile sourceFiles.map(_.fullPath).toList

The 2.10 RC1 compiler works for like three minutes then crashes, whereas 2.10 infinitely does something (full CPU usage). When I invoke the compiler via SBT (rather than programmatically) it works fine and compiles within less than a minute.

The shortened output looks like this (verbose - and running three minutes between the first line and the error):

[loaded class file C:\Program Files\scala\lib\scala-library.jar(scala/collection/mutable/StringBuilder.class) in 3ms]

Scala 2.10 stable

No further output. 100% CPU usage of 1 Core.

Scala 2.10 RC1

With RC1 I get this error after approximately 3 minutes:

error: 
     while compiling: Foo.scala
        during phase: typer
     library version: version 2.10.0-RC1
    compiler version: version 2.10.0-RC1
  reconstructed args:

Next piece of output (and final output before my application crashes) is an OutOfMemoryError. I'm not sure whether its cause is the code itself or the compile error. Both options appear strange to me, as it compiles on the SBT console and a compiler error should not consume that much memory, should it?

uncaught exception during compilation: java.lang.OutOfMemoryError
[error] (run-main) java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
    at scala.reflect.internal.Symbols$Symbol.createRefinementClassSymbol(Symbols.scala:1068)
    at scala.reflect.internal.Symbols$Symbol.newRefinementClass(Symbols.scala:406)
    at scala.reflect.internal.Types$class.refinedType(Types.scala:3504)
    at scala.reflect.internal.SymbolTable.refinedType(SymbolTable.scala:12)
    at scala.reflect.internal.Types$Type.narrow(Types.scala:459)
    at scala.reflect.internal.Types$class.specializedBy$1(Types.scala:6125)
    at scala.reflect.internal.Types$class.specializesSym(Types.scala:6129)
    at scala.reflect.internal.SymbolTable.specializesSym(SymbolTable.scala:12)
    at scala.reflect.internal.Types$$anonfun$thirdTry$1$2.apply(Types.scala:6021)
    at scala.reflect.internal.Types$$anonfun$thirdTry$1$2.apply(Types.scala:6021)
    at scala.collection.Iterator$class.forall(Iterator.scala:739)
    at scala.collection.AbstractIterator.forall(Iterator.scala:1156)
    at scala.collection.IterableLike$class.forall(IterableLike.scala:75)
    at scala.reflect.internal.Scopes$Scope.forall(Scopes.scala:44)
    at scala.reflect.internal.Types$class.thirdTry$1(Types.scala:6021)
    at scala.reflect.internal.Types$class.secondTry$1(Types.scala:5982)
    at scala.reflect.internal.Types$class.firstTry$1(Types.scala:5958)
    at scala.reflect.internal.Types$class.isSubType2(Types.scala:6101)
    at scala.reflect.internal.Types$class.isSubType(Types.scala:5710)
    at scala.reflect.internal.SymbolTable.isSubType(SymbolTable.scala:12)
    at scala.reflect.internal.Types$class.thirdTry$1(Types.scala:6043)
    at scala.reflect.internal.Types$class.secondTry$1(Types.scala:5982)
    at scala.reflect.internal.Types$class.firstTry$1(Types.scala:5958)
    at scala.reflect.internal.Types$class.isSubType2(Types.scala:6101)
    at scala.reflect.internal.Types$class.isSubType(Types.scala:5710)
    at scala.reflect.internal.SymbolTable.isSubType(SymbolTable.scala:12)
    at scala.reflect.internal.Types$class.scala$reflect$internal$Types$$specializesSym(Types.scala:6142)
    at scala.reflect.internal.Types$class.specializedBy$1(Types.scala:6125)
    at scala.reflect.internal.Types$class.specializesSym(Types.scala:6129)
    at scala.reflect.internal.SymbolTable.specializesSym(SymbolTable.scala:12)
    at scala.reflect.internal.Types$$anonfun$thirdTry$1$2.apply(Types.scala:6021)
    at scala.reflect.internal.Types$$anonfun$thirdTry$1$2.apply(Types.scala:6021)
[trace] Stack trace suppressed: run 'last compile:run' for the full output.
java.lang.RuntimeException: Nonzero exit code: 1
    at scala.sys.package$.error(package.scala:27)

I stumbled across Why am I getting OutOfMemoryError compilation error in Scala?. I am, however, not sure whether I'm actually simply lacking heap space for the compilation. There is no Maven involved, it's only Scala code and some JARs on the local build path.

I'm looking for the cause of the OutOfMemory error or a tweak to fix the error.

Answer 1

Using jvisualvm.exe (in the JDK) we found out that the compiler was indeed running low on memory. The GC was working too hard freeing memory, so it looked like an infinite loop (to be precise: happened when a symbol table's HashSet was enlarged).

Increasing the heap size to 2GB fixed the issue here.