How are closures implemented in scala?

Question

How are the variables outside of the scope of the function pulled into the function when it's created? I tried decompiling, but I had trouble understanding it. It looked like it uses putfield. Does putfield make a pointer to an object reference?

Answer 1

The answer is "it depends". There will probably be some major changes to this with the scala 2.11 release. Hopefully 2.11 will be able to inline simple closures.

But anyway, let's try to give an answer for the current scala version (javap below is from scala 2.10.2). Below is a very simple closure that uses a val and a var, and the javap output of the generated closure class. As you can see there is a major difference if you capture a var or if you capture a val.

If you capture a val it just gets passed to the closure class as a copy (you can do this since it is a val).

If you capture a var, the declaration of the var itself has to be changed at the call site location. Instead of a local int that sits on the stack, the var gets turned into an object of type scala.runtime.IntRef. This is basically just a boxed integer, but with a mutable int field.

(This is somewhat similar to the java approach of using a final array of size 1 when you want to update a field from inside an anonymous inner class)

This has some impact on performance. When you use a var in a closure, you have to generate the closure object and also the xxxRef object to contain the var. One mean thing is that if you have a block of code like this:

var counter = 0
// some large loop that uses the counter

And add a closure that captures counter somewhere else, the performance of your loop will be lowered significantly.

So the bottom line is: capturing vals is usually not a big deal, but be very careful with capturing vars.

---

object ClosureTest extends App {
  def test() {
    val i = 3
    var j = 0
    val closure:() => Unit = () => {
      j = i
    }
    closure()
  }
  test()
}

And here is the javap code of the generated closure class:

public final class ClosureTest$$anonfun$1 extends scala.runtime.AbstractFunction0$mcV$sp implements scala.Serializable {
  public static final long serialVersionUID;

  public final void apply();
Code:
  0: aload_0       
  1: invokevirtual #24                 // Method apply$mcV$sp:()V
  4: return        

  public void apply$mcV$sp();
Code:
  0: aload_0       
  1: getfield      #28                 // Field j$1:Lscala/runtime/IntRef;
  4: aload_0       
  5: getfield      #30                 // Field i$1:I
  8: putfield      #35                 // Field scala/runtime/IntRef.elem:I
  11: return        

  public final java.lang.Object apply();
Code:
  0: aload_0       
  1: invokevirtual #38                 // Method apply:()V
  4: getstatic     #44                 // Field scala/runtime/BoxedUnit.UNIT:Lscala/runtime/BoxedUnit;
  7: areturn       

  public ClosureTest$$anonfun$1(int, scala.runtime.IntRef);
Code:
  0: aload_0       
  1: iload_1       
  2: putfield      #30                 // Field i$1:I
  5: aload_0       
  6: aload_2       
  7: putfield      #28                 // Field j$1:Lscala/runtime/IntRef;
  10: aload_0       
  11: invokespecial #48                 // Method scala/runtime/AbstractFunction0$mcV$sp."<init>":()V
  14: return        
}

Answer 2

Here is the Bytecode for the following method containing a closure:

def run()
{
val buff = new ArrayBuffer[Int]();

val i = 7;

buff.foreach( a =>  {  a + i  }  )
}

Bytecode:

  public class com.anarsoft.plugin.scala.views.ClosureTest$$anonfun$run$1 extends   scala.runtime.AbstractFunction1$mcII$sp implements scala.Serializable {

// Field descriptor #14 J
public static final long serialVersionUID = 0L;

// Field descriptor #18 I
private final int i$1;

public ClosureTest$$anonfun$run$1(com.anarsoft.plugin.scala.views.ClosureTest $outer, int i$1);

...

The Compiler generates a new ClosureTest$$anonfun$run$1 with a constructor with two fields for the variables outside the scope, e.g. i and this of the calling class.

Answer 3

Let's see a concrete example :

scala> var more = 1
more: Int = 1

scala> val f = (x: Int) => x + more
f: Int => Int = <function1>

This closure is an open term.

scala> f(1)
res38: Int = 2

scala> more = 2
more: Int = 2

scala> f(1)
res39: Int = 3

As you see, the closure contains a reference to the captured more variable