Scala 2.8 vs 2.9 benchmark, strange results

Question

Some time ago, I wrote some code to decide which method of updating mutable variable with new value (but without creating new objects) is faster. One method uses a temporary value and there is an extra copying, and other does not.

code sample #1 - with reassignment

class Test(var x: Float, var y: Float) {
    @inline final def :=(x: Float, y: Float) = {
        this.x = x
        this.y = y
    }
    //...
    @inline final def :=(o: Test) = { //reassign from arg "o"
        x = o.x
        y = o.y
    }
    //...
}
object Benchmark {
    //...
    val tmp = new Test(0, 0)
    @inline final def calc_something_for_reassign(a: Float, b: Float) = {
        tmp := (a, b)
        tmp.something_m //this is a simple method that changes the object
        tmp
    }
    //...
}

code sample #2 - without reassignment

class Test(var x: Float, var y: Float) {
    @inline final def := (x: Float, y: Float) = //it's the same as in sample #1
    //...
    @inline final def := (of: (Test) => Unit) = { //apply mutating function "of"
        of(this)
    }
//...
}
object Benchmark {
    //...
    @inline final def calc_something_for_forwarding(a: Float, b: Float) = {
        (result: Test) => {
            result := (a, b)
            result.something_m
        }
    }
}

Full code is here: http://ideone.com/A62Ts

On my PC (scala compiler v. 2.8, jre7) the results looks like expected:

reassignment: 0.046sec; Result:42.0, 3.0
forwarding: 0.007sec; Result:42.0, 3.0
forwarding: 0.006sec; Result:42.0, 3.0
reassignment: 0.044sec; Result:42.0, 3.0

(Yes, if I increase test length, the ratio is the same, and compiler options don't matter.)

I executed this test on a cell phone with Android OS 2.3.4 and winner is still the same. But, as we can see, on Ideone.com opposite things happen, second method is much slower. So can you explain, what is actually happening there? And which (of these two) version of Scala compiler is working right? Is it a bug in scala-2.9.1?

UPDATE: on scala 2.9 test prints the following (tried many times, nothing changes too much):

reassignment: 0.047sec; Result:42.0, 3.0
forwarding: 0.032sec; Result:42.0, 3.0
forwarding: 0.219sec; Result:42.0, 3.0
reassignment: 0.045sec; Result:42.0, 3.0

And with any number of cycles the third run is longer than all others. May it cause GC? Or why else such weird thing can happen? If I instead call them in order b, b, a, b, b, a, then there's no such strange peaks:

println(Benchmark.run_b(Int.MaxValue/10))
println(Benchmark.run_b(Int.MaxValue/10))
println(Benchmark.run_a(Int.MaxValue/10))
println(Benchmark.run_b(Int.MaxValue/10))
println(Benchmark.run_b(Int.MaxValue/10))
println(Benchmark.run_a(Int.MaxValue/10))

Results in scala 2.8 (stable):

forwarding: 0.012sec; Result:42.0, 3.0
forwarding: 0.012sec; Result:42.0, 3.0
reassignment: 0.347sec; Result:42.0, 3.0
forwarding: 0.011sec; Result:42.0, 3.0
forwarding: 0.005sec; Result:42.0, 3.0
reassignment: 0.333sec; Result:42.0, 3.0

Results in scala 2.9 (stable):

forwarding: 0.184sec; Result:42.0, 3.0
forwarding: 0.179sec; Result:42.0, 3.0
reassignment: 0.354sec; Result:42.0, 3.0
forwarding: 0.17sec; Result:42.0, 3.0
forwarding: 0.169sec; Result:42.0, 3.0
reassignment: 0.342sec; Result:42.0, 3.0

However, the method with reassignment in scala 2.9 is equally long as in scala 2.8, but fast method is about 10x slower in scala 2.9 (but still about 2 times faster than slow method). So the original question is still alive, why it's so much slower in scala 2.9.

Answer 1

For one thing, the ideone runtime will be memory resident and the JVM may have some stats skewed by other users which has caused it to construct some slightly bizarre native code. I'd be tempted to ignore anything that comes out of it performance wise as there's so many things that are unknown which affect benchmarks. It's even possible they've got a modified version to suit their needs more specifically.