How to learn about using scala.None from Java using javap?

Question

In a previous question, Accessing scala.None from Java, it seems that people had used javap to figure out how to access scala.None from Java. I would like to know how they did that. FYI, the answer is:

scala.Option$.MODULE$.apply(null);

which can be shorted to:

scala.Option.apply(null);

Given this program (OptionTest.scala):

object OptionTest extends App {
  val x = scala.None
  val y = scala.Some("asdf")
}

I ran javap on it like this:

javap -s -c -l -private OptionTest

This is a portion of the javap output:

public static final scala.None$ x();
  Signature: ()Lscala/None$;
  Code:
   0:   getstatic  #11; //Field OptionTest$.MODULE$:LOptionTest$;
   3:   invokevirtual  #55; //Method OptionTest$.x:()Lscala/None$;
   6:   areturn

I also ran javap on scala.None and scala.Option. How would one figure out from the javap output that:

1. None is a sole object of None.type type which extends Option
2. The apply() method for the companion object is required

?

Answer 1

There are rules how Scala code is compiled to JVM-bytecode. Because of potential name clashes the generated code is not always intuitive to understand but if the rules are known it is possible to get access to the compiled Scala code within Java.

Attention: While writing this, I noticed that javac and eclipse-javac behave differently in accessing Scala code from Java. It is possible that the code below compile with one of them but not with the other.

Classes, Constructors, Methods

There are no special rules here. The following Scala class

class X(i: Int) {
  def m1 = i*2
  def m2(a: Int)(b: Int) = a*b
  def m3(a: Int)(implicit b: Int) = a*b
}

can be accessed like a normal Java class. It is compiled to a file named X.class:

X x = new X(7);
x.m1();
x.m2(3, 5);
x.m3(3, 5);

Notice, that for methods without a parameterlist an empty parameterlist is created. Multiple parameterlists are merged to a single one.

Fields, Values

For a class class X(var i: Int) Getters and Setters are created. For a class class X(val i: Int) only a Getter is created:

//Scala
val x = new X(5)
x.i = 3 // Setter
x.i // Getter

//Java
X x = new X(5);
x.i_$eq(3); // Setter
x.i(); // Getter

Notice, that in Java an identifier is not allowed to include special signs. Therefore scalac generates for each of these special signs a specific name. There is a class scala.reflect.NameTransformer which can encode/decode the ops:

scala> import scala.reflect.NameTransformer._
import scala.reflect.NameTransformer._

scala> val ops = "~=<>!#%^&|*/+-:\\?@"
ops: String = ~=<>!#%^&|*/+-:\?@

scala> ops map { o => o -> encode(o.toString) } foreach println
(~,$tilde)
(=,$eq)
(<,$less)
(>,$greater)
(!,$bang)
(#,$hash)
(%,$percent)
(^,$up)
(&,$amp)
(|,$bar)
(*,$times)
(/,$div)
(+,$plus)
(-,$minus)
(:,$colon)
(\,$bslash)
(?,$qmark)
(@,$at)

A class class X { var i = 5 } is translated by the same schema as when the field is created in the constructor. Direct access to the variable i from Java is not possible, because it is private.

Objects

There is no such thing as a Scala object in Java. Therefore scalac has to do some magic. For an object object X { val i = 5 } two JVM-class files are generated: X.class and X$.class. The first one works like an interface, it includes static methods to access fields and methods of the Scala object. The latter is a singleton class which cannot be instantiated. It has a Field which holds the singleton instance of the class, named MODULE$, which allows access to the singleton:

X.i();
X$.MODULE$.i();

Case classes

The Scala compiler automatically generates an apply-method for a case class and Getters for fields. The case class case class X(i: Int) is easily accessed:

new X(3).i();
X$.MODULE$.apply(3);

Traits

A trait trait T { def m }, which contains only abstract members, is compiled to an interface, which is placed in a class files named T.class. Therefore it can easily implemented by a Java class:

class X implements T {
  public void m() {
    // do stuff here
  }
}

If the trait contains concrete members there is a class file named <trait_name>$class.class generated, additionally to the normal interface. The trait

trait T {
  def m1
  def m2 = 5
}

can also easily implemented within Java. The class file T$class.class contains the concrete members of the trait, but it seems that they are impossible to access from Java. Neither javac nor the eclipse-javac will compile an access to this class.

Some more detail about how traits are compiled can be found here.

Functions

Function literals are compiled as anonymous instances of the classes FunctionN. A Scala object

object X {
  val f: Int => Int = i => i*2
  def g: Int => Int = i => i*2
  def h: Int => Int => Int = a => b => a*b
  def i: Int => Int => Int = a => {
    def j: Int => Int = b => a*b
    j
  }
}

is compiled to the normal class-files, as describes above. Furthermore each function literal gets its own class-file. So, for function values a class file named <class_name>$$anonfun$<N>.class is generated, where N is a continuous number. For function methods (methods, which return a function) a class file named <class_name>$$anonfun$<method_name>$<N>.class is generated. The parts of the function name are separated by dollar signs and in front of the anonfun identifier there are also two dollar signs. For nested functions the name of the nested function is appended to the outer function, this means an inner function will get a class file like <class_name>$$anonfun$<outer_method_name>$<N>$$anonfun$<inner_method_name>$<N>.class. When an inner function does not have a name, as seen in h it gets the name apply.

This means in our case we get:

• X$$anonfun$1.class for f
• X$$anonfun$g$1.class for g
• X$$anonfun$h$1$$anonfun$apply$1.class for h
• X$$anonfun$i$1.class and X$$anonfun$i$1$$anonfun$j$1$1.class for i and j

To access them use their apply-method:

X.f().apply(7);
X.g().apply(7);
X.h().apply(3).apply(5);
X.i().apply(3).apply(5);

Answer the question

You should know:

• a normal Scala class can accessed by their constructors or their apply-methods
• when there is no constructor than there is an apply-method
• when there is no constructor and no apply method than there is a another class file named the same way the class is called which appends a dollar sign at the end. Search this class for a MODULE$ field
• constructors and apply-methods are inherited, so search the super-classes if you can't find anything in the subclasses

Some examples

Option

// javap scala.Option
public abstract class scala.Option extends java.lang.Object implements ... {
  ...
  public static final scala.Option apply(java.lang.Object);
  public scala.Option();
}

javap says it has a constructor and an apply method. Furthermore it says the class is abstract. Thus only the apply-method can used:

Option.apply(3);

Some

// javap scala.Some
public final class scala.Some extends scala.Option implements ... {
  ...
  public scala.Some(java.lang.Object);
}

It has a constructor and an apply-method (because we know Option has one and Some extends Option). Use one of them and be happy:

new Some<Integer>(3);
Some.apply(3);

None

// javap scala.None
public final class scala.None extends java.lang.Object{
  ...
}

It has no constructor, no apply-method and doesn't extend Option. So, we will take a look to None$:

// javap -private scala.None$
public final class scala.None$ extends scala.Option implements ... {
  ...
  public static final scala.None$ MODULE$;
  private scala.None$();
}

Yeah! We found a MODULE$ field and the apply-method of Option. Furthermore we found the private constructor:

None$.apply(3) // returns Some(3). Please use the apply-method of Option instead
None$.MODULE$.isDefined(); // returns false
new None$(); // compiler error. constructor not visible

List

scala.collection.immutable.List is abstract, thus we have to use scala.collection.immutable.List$. It has an apply-method which expects an scala.collection.Seq. So to get a List we need first a Seq. But if we look to Seq there is no apply-method. Furthermore when we look at the super-classes of Seq and at scala.collection.Seq$ we can only find an apply-methods which expects a Seq. So, what to do?

We have to take a look how scalac creates an instance of List or Seq. First create a Scala class:

class X {
  val xs = List(1, 2, 3)
}

Compile it with scalac and look at the class file with javap:

// javap -c -private X
public class X extends java.lang.Object implements scala.ScalaObject{
...
public X();
  Code:
   0:   aload_0
   1:   invokespecial   #20; //Method java/lang/Object."<init>":()V
   4:   aload_0
   5:   getstatic   #26; //Field scala/collection/immutable/List$.MODULE$:Lscala/collection/immutable/List$;
   8:   getstatic   #31; //Field scala/Predef$.MODULE$:Lscala/Predef$;
   11:  iconst_3
   12:  newarray int
   14:  dup
   15:  iconst_0
   16:  iconst_1
   17:  iastore
   18:  dup
   19:  iconst_1
   20:  iconst_2
   21:  iastore
   22:  dup
   23:  iconst_2
   24:  iconst_3
   25:  iastore
   26:  invokevirtual   #35; //Method scala/Predef$.wrapIntArray:([I)Lscala/collection/mutable/WrappedArray;
   29:  invokevirtual   #39; //Method scala/collection/immutable/List$.apply:(Lscala/collection/Seq;)Lscala/collection/immutable/List;
   32:  putfield    #13; //Field xs:Lscala/collection/immutable/List;
   35:  return

}

The constructor is interesting. It tells us, that an array of ints is created (l. 12) which is filled with 1, 2 and 3. (l. 14-25). After that this array is delivered to scala.Predef$.wrapIntArray (l. 26). This resulting scala.collection.mutable.WrappedArray is again delivered to our List (l. 29). At the end, the List is stored in the field (l. 32). When we wanna create a List in Java, we have to do the same:

int[] arr = { 1, 2, 3 };
WrappedArray<Object> warr = Predef$.MODULE$.wrapIntArray(arr);
List$.MODULE$.apply(warr);

// or shorter
List$.MODULE$.apply(Predef$.MODULE$.wrapIntArray(new int[] { 1, 2, 3 }));

This looks ugly, but it works. If you create a nice looking library which wraps the access to the Scala library it will be easy to use Scala from Java.

Summary

I know there are some more rules how Scala code is compiled to bytecode. But I think with the information above it should be possible to find these rules by yourself.

Answer 2

I'm not competing with the other answer, but since people seem often not to notice this, you can do this in the repl.

scala> :paste
// Entering paste mode (ctrl-D to finish)

object OptionTest extends App {
  val x = scala.None
  val y = scala.Some("asdf")
}

// Exiting paste mode, now interpreting.

defined module OptionTest

scala> :javap -v OptionTest$
Compiled from "<console>"
public final class OptionTest$ extends java.lang.Object implements scala.App,scala.ScalaObject
  SourceFile: "<console>"
  Scala: length = 0x

  [lots of output etc]   

  public scala.None$ x();
    Code:
     Stack=1, Locals=1, Args_size=1
     0: aload_0
     1: getfield    #65; //Field x:Lscala/None$;
     4: areturn