Java Condition Operators Speed vs Versions of Hotspot VM

Question

Please consider the following java source:

package com.stackoverflow;

 public class CondSpeed {
 private static final long COUNT = 1000000000;
 private static final long OUTER_COUNT = 15;

 private static long notEqOperator = 0L;
  private static long notOperator = 0L;
private static long equalsFalse = 0L;

public CondSpeed() {
 super();
}

public static void main(String[] args) {

 for(int outCount = 0;outCount < OUTER_COUNT;outCount++){
  notEqOperator += testNotEaualsOperator();
  equalsFalse += testEqualFalse();
  notOperator += testNotOperator();
 }

 long avrForNotEqOperator = (notEqOperator / OUTER_COUNT);
 long avrForEqualsFalse = (equalsFalse / OUTER_COUNT);
 long avrForNotOperator = (notOperator / OUTER_COUNT);

 System.out.println("Avr for Not Equals Operator: "+avrForNotEqOperator);
 System.out.println("Avr for Equals \"false\" Operator: "+avrForEqualsFalse);
 System.out.println("Avr for Not Operator: "+avrForNotOperator);

}

private static long testEqualFalse(){
 long now = System.currentTimeMillis();

 for(long i = 0;i < COUNT;i++){
  boolean truFalse = returnTrueOrFalse();

  if(truFalse == false){
   //do nothing...
  }
 }

 return (System.currentTimeMillis() - now);
}


   private static long testNotOperator(){
 long now = System.currentTimeMillis();

 for(long i = 0;i < COUNT;i++){
  boolean truFalse = returnTrueOrFalse();

  if(!truFalse){
//do nothing...
  }
 }

 return (System.currentTimeMillis() - now);
}

private static long testNotEaualsOperator(){
 long now = System.currentTimeMillis();

 for(long i = 0;i < COUNT;i++){
  boolean truFalse = returnTrueOrFalse();

  if(truFalse != true){
   //do nothing...
  }
 }

 return (System.currentTimeMillis() - now);
}

private static boolean isFalse;
private static boolean returnTrueOrFalse(){
 if(isFalse){
  isFalse = false;
 }
 else{
  isFalse = true;
 }
 return isFalse;
}

}

As you can see it is a test against 3 versions of if(false) conditions.

• I am interested in why the results are firstly different in the various condition statements. ((I know that its obviously the way the compiler interpretes the .java into bytecode.)) Is there more to it than this?
• Secondly. Look at the differences in different Hotspot VMs. See bottom. Is this because of updates/improvements the the VM over versions? Or is there more to it?
• Is this the best way to test something like this?

---Results Mac OS X---

JavaVM HotSpot 1.6.0

Avr for Not Equals Operator: 1937
Avr for Equals "false" Operator: 1937
Avr for Not Operator: 1941

JavaVM HotSpot 1.5.0

Avr for Not Equals Operator: 5023
Avr for Equals "false" Operator: 5035
Avr for Not Operator: 5067

JavaVM HotSpot 1.4.2

Avr for Not Equals Operator: 3993
Avr for Equals "false" Operator: 4015
Avr for Not Operator: 4009

JavaVM HotSpot 1.4.0

Avr for Not Equals Operator: 3961
Avr for Equals "false" Operator: 3960
Avr for Not Operator: 3961

Thanks.

Answer 1

The differences between !, != and == look like random noise - were you really expecting them to come out to exactly the same number of millisecs?

The improvement with JVM version, however, is almost certainly real, though it is likely very specific to that particular piece of code, a matter of something falling within a complexity threshold to be handled properly. Something even slightly different might not show the same results.

To improve testing, calculate the standard deviation of each test run and see if they are statistically different (or just print out all 10 results and eyeball them).

Answer 2

Microbenchmarks like this don't tell you anything interesting. You are doing a billion iterations of the test and the result comes back in seconds. How many cycles per iteration is that? The microbenchmark isn't doing any work.

Answer 3

If the JVM is doing a decent job it will detect that the following statements do nothing that affects the computation and will completely optimize them away.

if (truFalse != true) {
     //do nothing...
}
...
if (truFalse == false) {
   //do nothing...
}
...
if (!truFalse) {
   //do nothing...
}

In other words, your benchmarks are probably not measuring anything different in the three cases.

Lessons to learn:

1. It is difficult to be sure that you are getting meaningful numbers from a microbenchmark.
2. The relative numbers can vary significantly from one JVM to another. A "clever trick" that helps on one JVM could actually get in the way on another.
3. A compiler is likely to do a far better job of micro-optimizing your Java programs for each platform than you are.

The best strategy is to leave micro-optimization to the compiler, and focus on the "macro" issues like using the best algorithm. Also, use a execution profiler to figure out where it is worthwhile spending time to optimize.