Testing initialization safety of final fields

Question

I am trying to simply test out the initialization safety of final fields as guaranteed by the JLS. It is for a paper I'm writing. However, I am unable to get it to 'fail' based on my current code. Can someone tell me what I'm doing wrong, or if this is just something I have to run over and over again and then see a failure with some unlucky timing?

Here is my code:

public class TestClass {

    final int x;
    int y;
    static TestClass f;

    public TestClass() {
        x = 3;
        y = 4;
    }

    static void writer() {
        TestClass.f = new TestClass();
    }

    static void reader() {
        if (TestClass.f != null) {
            int i = TestClass.f.x; // guaranteed to see 3
            int j = TestClass.f.y; // could see 0

            System.out.println("i = " + i);
            System.out.println("j = " + j);
        }
    }
}

and my threads are calling it like this:

public class TestClient {

    public static void main(String[] args) {

        for (int i = 0; i < 10000; i++) {
            Thread writer = new Thread(new Runnable() {
                @Override
                public void run() {
                    TestClass.writer();
                }
            });

            writer.start();
        }

        for (int i = 0; i < 10000; i++) {
            Thread reader = new Thread(new Runnable() {
                @Override
                public void run() {
                    TestClass.reader();
                }
            });

            reader.start();
        }
    }
}

I have run this scenario many, many times. My current loops are spawning 10,000 threads, but I've done with this 1000, 100000, and even a million. Still no failure. I always see 3 and 4 for both values. How can I get this to fail?

Answer 1

From Java 5.0, you are guarenteed that all threads will see the final state set by the constructor.

If you want to see this fail, you could try an older JVM like 1.3.

I wouldn't print out every test, I would only print out the failures. You could get one failure in a million but miss it. But if you only print failures, they should be easy to spot.

A simpler way to see this fail is to add to the writer.

f.y = 5;

and test for

int y = TestClass.f.y; // could see 0, 4 or 5
if (y != 5)
    System.out.println("y = " + y);

Answer 2

I wrote the spec. The TL; DR version of this answer is that just because it may see 0 for y, that doesn't mean it is guaranteed to see 0 for y.

In this case, the final field spec guarantees that you will see 3 for x, as you point out. Think of the writer thread as having 4 instructions:

r1 = <create a new TestClass instance>
r1.x = 3;
r1.y = 4;
f = r1;

The reason you might not see 3 for x is if the compiler reordered this code:

r1 = <create a new TestClass instance>
f = r1;
r1.x = 3;
r1.y = 4;

The way the guarantee for final fields is usually implemented in practice is to ensure that the constructor finishes before any subsequent program actions take place. Imagine someone erected a big barrier between r1.y = 4 and f = r1. So, in practice, if you have any final fields for an object, you are likely to get visibility for all of them.

Now, in theory, someone could write a compiler that isn't implemented that way. In fact, many people have often talked about testing code by writing the most malicious compiler possible. This is particularly common among the C++ people, who have lots and lots of undefined corners of their language that can lead to terrible bugs.

Answer 3

I'd like to see a test which fails or an explanation why it's not possible with current JVMs.

Multithreading and Testing

You can't prove that a multithreaded application is broken (or not) by testing for several reasons:

• the problem might only appear once every x hours of running, x being so high that it is unlikely that you see it in a short test
• the problem might only appear with some combinations of JVM / processor architectures

In your case, to make the test break (i.e. to observe y == 0) would require the program to see a partially constructed object where some fields have been properly constructed and some not. This typically does not happen on x86 / hotspot.

How to determine if a multithreaded code is broken?

The only way to prove that the code is valid or broken is to apply the JLS rules to it and see what the outcome is. With data race publishing (no synchronization around the publication of the object or of y), the JLS provides no guarantee that y will be seen as 4 (it could be seen with its default value of 0).

Can that code really break?

In practice, some JVMs will be better at making the test fail. For example some compilers (cf "A test case showing that it doesn't work" in this article) could transform TestClass.f = new TestClass(); into something like (because it is published via a data race):

(1) allocate memory
(2) write fields default values (x = 0; y = 0) //always first
(3) write final fields final values (x = 3)    //must happen before publication
(4) publish object                             //TestClass.f = new TestClass();
(5) write non final fields (y = 4)             //has been reodered after (4)

The JLS mandates that (2) and (3) happen before the object publication (4). However, due to the data race, no guarantee is given for (5) - it would actually be a legal execution if a thread never observed that write operation. With the proper thread interleaving, it is therefore conceivable that if reader runs between 4 and 5, you will get the desired output.

I don't have a symantec JIT at hand so can't prove it experimentally :-)

Answer 4

Here is an example of default values of non final values being observed despite that the constructor sets them and doesn't leak this. This is based off my other question which is a bit more complicated. I keep seeing people say it can't happen on x86, but my example happens on x64 linux openjdk 6...

Answer 5

What about you modified the constructor to do this:

public TestClass() {
 Thread.sleep(300);
   x = 3;
   y = 4;
}

I am not an expert on JLF finals and initializers, but common sense tells me this should delay setting x long enough for writers to register another value?

Answer 6

What if one changes the scenario into

public class TestClass {

    final int x;
    static TestClass f;

    public TestClass() {
        x = 3;
    }

    int y = 4;

    // etc...

}

?

Answer 7

Better understanding of why this test does not fail can come from understanding of what actually happens when constructor is invoked. Java is a stack-based language. TestClass.f = new TestClass(); consists of four action. First new instruction is called, its like malloc in C/C++, it allocates memory and places a reference to it on the top of the stack. Then reference is duplicated for invoking a constructor. Constructor in fact is like any other instance method, its invoked with the duplicated reference. Only after that reference is stored in the method frame or in the instance field and becomes accessible from anywhere else. Before the last step reference to the object is present only on the top of creating thread's stack and no body else can see it. In fact there is no difference what kind of field you are working with, both will be initialized if TestClass.f != null. You can read x and y fields from different objects, but this will not result in y = 0. For more information you should see JVM Specification and Stack-oriented programming language articles.

UPD: One important thing I forgot to mention. By java memory there is no way to see partially initialized object. If you do not do self publications inside constructor, sure.

JLS:

An object is considered to be completely initialized when its constructor finishes. A thread that can only see a reference to an object after that object has been completely initialized is guaranteed to see the correctly initialized values for that object's final fields.

JLS:

There is a happens-before edge from the end of a constructor of an object to the start of a finalizer for that object.

Broader explanation of this point of view:

It turns out that the end of an object's constructor happens-before the execution of its finalize method. In practice, what this means is that any writes that occur in the constructor must be finished and visible to any reads of the same variable in the finalizer, just as if those variables were volatile.

UPD: That was the theory, let's turn to practice.

Consider the following code, with simple non-final variables:

public class Test {

    int myVariable1;
    int myVariable2;

    Test() {
        myVariable1 = 32;
        myVariable2 = 64;
    }

    public static void main(String args[]) throws Exception {
        Test t = new Test();
        System.out.println(t.myVariable1 + t.myVariable2);
    }
}

The following command displays machine instructions generated by java, how to use it you can find in a wiki:

java.exe -XX:+UnlockDiagnosticVMOptions -XX:+PrintAssembly -Xcomp -XX:PrintAssemblyOptions=hsdis-print-bytes -XX:CompileCommand=print,*Test.main Test

It's output:

...
0x0263885d: movl   $0x20,0x8(%eax)    ;...c7400820 000000
                                    ;*putfield myVariable1
                                    ; - Test::<init>@7 (line 12)
                                    ; - Test::main@4 (line 17)
0x02638864: movl   $0x40,0xc(%eax)    ;...c7400c40 000000
                                    ;*putfield myVariable2
                                    ; - Test::<init>@13 (line 13)
                                    ; - Test::main@4 (line 17)
0x0263886b: nopl   0x0(%eax,%eax,1)   ;...0f1f4400 00
...

Field assignments are followed by NOPL instruction, one of it's purposes is to prevent instruction reordering.

Why does this happen? According to specification finalization happens after constructor returns. So GC thread cant see a partially initialized object. On a CPU level GC thread is not distinguished from any other thread. If such guaranties are provided to GC, than they are provided to any other thread. This is the most obvious solution to such restriction.

Results:

1) Constructor is not synchronized, synchronization is done by other instructions.

2) Assignment to object's reference cant happen before constructor returns.

Answer 8

What's going on in this thread? Why should that code fail in the first place?

You launch 1000s of threads that will each do the following:

TestClass.f = new TestClass();

What that does, in order:

1. evaluate TestClass.f to find out its memory location
2. evaluate new TestClass(): this creates a new instance of TestClass, whose constructor will initialize both x and y
3. assign the right-hand value to the left-hand memory location

An assignment is an atomic operation which is always performed after the right-hand value has been generated. Here is a citation from the Java language spec (see the first bulleted point) but it really applies to any sane language.

This means that while the TestClass() constructor is taking its time to do its job, and x and y could conceivably still be zero, the reference to the partially initialized TestClass object only lives in that thread's stack, or CPU registers, and has not been written to TestClass.f

Therefore TestClass.f will always contain:

• either null, at the start of your program, before anything else is assigned to it,
• or a fully initialized TestClass instance.