Why is javac 1.5 running so slowly compared with the Eclipse compiler?

Question

I have a Java Maven project with about 800 source files (some generated by javacc/JTB) which is taking a good 25 minutes to compile with javac.

When I changed my pom.xml over to use the Eclipse compiler, it takes about 30 seconds to compile.

Any suggestions as to why javac (1.5) is running so slowly? (I don't want to switch over to the Eclipse compiler permanently, as the plugin for Maven seems more than a little buggy.)

I have a test case which easily reproduces the problem. The following code generates a number of source files in the default package. If you try to compile ImplementingClass.java with javac, it will seem to pause for an inordinately long time.

import java.io.File;
import java.io.FileNotFoundException;
import java.io.PrintStream;

public class CodeGenerator
{
    private final static String PATH = System.getProperty("java.io.tmpdir");
    private final static int NUM_TYPES = 1000;

    public static void main(String[] args) throws FileNotFoundException
    {
        PrintStream interfacePs = new PrintStream(PATH + File.separator + "Interface.java");
        PrintStream abstractClassPs = new PrintStream(PATH + File.separator + "AbstractClass.java");
        PrintStream implementingClassPs = new PrintStream(PATH + File.separator + "ImplementingClass.java");
        interfacePs.println("public interface Interface<T> {");
        abstractClassPs.println("public abstract class AbstractClass<T> implements Interface<T> {");
        implementingClassPs.println("public class ImplementingClass extends AbstractClass<Object> {");

        for (int i=0; i<NUM_TYPES; i++)
        {
            String nodeName = "Node" + i;
            PrintStream nodePs = new PrintStream(PATH + File.separator + nodeName + ".java");
            nodePs.printf("public class %s { }\n", nodeName);
            nodePs.close();
            interfacePs.printf("void visit(%s node, T obj);%n", nodeName);
            abstractClassPs.printf("public void visit(%s node, T obj) { System.out.println(obj.toString()); }%n", nodeName);
        }
        interfacePs.println("}");
        abstractClassPs.println("}");
        implementingClassPs.println("}");
        interfacePs.close();
        abstractClassPs.close();
        implementingClassPs.close();
    }
}

Answer 1

You get the same behaviour with JDK 1.6, including update 14, build 04, using G1 doesn't change the behaviour, (though G1 appears to work really well).

Monitoring javac with jvisualvm, repeated thread dumps show the main thread spending lots of time in

at com.sun.tools.javac.code.Types.isSubSignature(Types.java:1846)
at com.sun.tools.javac.code.Symbol$MethodSymbol.overrides(Symbol.java:1108)
at com.sun.tools.javac.code.Symbol$MethodSymbol.implementation(Symbol.java:1159)
at com.sun.tools.javac.comp.Check.checkCompatibleConcretes(Check.java:1239)
at com.sun.tools.javac.comp.Check.checkCompatibleSupertypes(Check.java:1567)
at com.sun.tools.javac.comp.Attr.attribClassBody(Attr.java:2674)
at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:2628)
at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:2564)
at com.sun.tools.javac.main.JavaCompiler.attribute(JavaCompiler.java:1036)
at com.sun.tools.javac.main.JavaCompiler.compile2(JavaCompiler.java:765)
at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:730)
at com.sun.tools.javac.main.Main.compile(Main.java:353)
at com.sun.tools.javac.main.Main.compile(Main.java:279)
at com.sun.tools.javac.main.Main.compile(Main.java:270)
at com.sun.tools.javac.Main.compile(Main.java:69)
at com.sun.tools.javac.Main.main(Main.java:54)

and churning through a large number of short lived instances of these classes:

com.sun.tools.javac.code.Types$Subst
com.sun.tools.javac.util.List
com.sun.tools.javac.code.Types$MethodType

I suspect the code is churning through com.sun.tools.javac.comp.Check.checkCompatibleConcretes comparing each method with every other method

That method's javadoc:

/** Check that a class does not inherit two concrete methods
 *  with the same signature.
 */

It may be that eclipse's compiler either doesn't perform that check, or doesn't perform it in the same way.

Answer 2

Sun has confirmed to me by email that this is a new bug (6827648 in their bug database).

Answer 3

It may be that the javac compiler operates close at its heap limit (64MB or so). In that case, it spends most of the time in the garbage collector. Give the compiler a good chunk of memory, say 256M or 512M and see if it runs faster.

Answer 4

The fact that you're using generated source, the massive difference in speed and the StackOverflowError might suggest that one (or more) of your files have some constructs that the javac parsers doesn't agree with.

Could you try to compile only subsets of your code and see if any one class/package slows down the process especially (probably one of the generated ones).

Answer 5

For the Sun compiler you are starting up a whole JVM process for each file you wish to compile. For the Eclipse compiler it is just connecting to a daemon process. I suggest setting fork to false, although it still may not be quite as fast.

Answer 6

Perhaps the Eclipse build is only compiling modified source. What happens if you compile it in eclipse after a clean?

Answer 7

I don't know how maven calls the compiler but the performance numbers you mention suggest that javac is executed in it's own process/VM as already suggested in another answer. As starting a new process/VM for every file you compile is very costly you need to ensure to configure the compiler to use the VM you might alreay have. I know ANT offers that, but I have not used maven myself. Given the fact that it is popular I doubt that it lacks such an important feature though.

Answer 8

I think something like the following is going on: Maven forks javac, JVM processes for separate steps in its life-cycle: Maven Build Life-cycle

Eclipse typically runs its compile in the background (on Save), so that step will be added to the compile phase. If there are substantial dependencies, this is where you're losing throughput.

In addition (depending upon mvn configuration) each test method gets its own JVM. Since test passage is a pre-req to the package phase, it's possible that you're losing time to your JUnit test executions (particularly if they're slow running tests). This is only a likely culprit if you have a lot of test code in your source tree.

Most likely of all, your class does substantial amounts of File I/O, so that's an area of opportunity. It looks like your loop is executing 1000 times per File discovery event meaning 800*1000 =800,000 PrintStream creations in the body of the loop.