XOR Java Neural Network

Question

Attempting a XOR neural network in java, but the network always predicts the final output it trains in.

Here is my code

for( int i = 0; i < 4; i++ ) { //Forward pass
        diff = 1;
        while( diff > 0.01 ) {
        SumError = 0;
        Y1 = ( InputOne[i] * Weight[0] ) + ( InputTwo[i] * Weight[1] ) + Weight[6];
        Y1 = 1 / ( 1 + Math.exp( -Y1 ) );
        Node1[i] = Y1;

        Y2 = ( InputOne[i] * Weight[2] ) + ( InputTwo[i] * Weight[3] ) + Weight[7];
        Y2 = 1 / ( 1 + Math.exp( -Y2 ) );
        Node2[i] = Y2;

        Y3 = ( Y1 * Weight[4] ) + ( Y2 * Weight[5] ) + Weight[8];
        Y3 = 1 / ( 1 + Math.exp( -Y3 ) );
        Node3[i] = Y3;

        diff = Math.abs( Result[i] - Y3 );

        System.out.println( i + " " + Result[i] + " " + Y3 + " " + diff );

        //Error Signals
        Delta3[i] = Y3 * ( 1 - Y3 ) * ( Result[i] - Y3 );
        Delta2[i] = Node2[i] * ( 1 - Node2[i] ) * ( Delta3[i] * Weight[5] );
        Delta1[i] = Node1[i] * ( 1 - Node1[i] ) * ( Delta3[i] * Weight[4] );

        //Update Weights
        Weight[0] = Weight[0] + ( ( WeightChange[0] * alpha ) + ( eta * Delta2[i] * InputOne[i] ) );
        Weight[2] = Weight[2] + ( ( WeightChange[2] * alpha ) + ( eta * Delta2[i] * InputTwo[i] ) );

        Weight[1] = Weight[1] + ( ( WeightChange[1] * alpha ) + ( eta * Delta1[i] * InputOne[i] ) );
        Weight[3] = Weight[3] + ( ( WeightChange[3] * alpha ) + ( eta * Delta1[i] * InputTwo[i] ) );

        Weight[4] = Weight[4] + ( ( WeightChange[4] * alpha ) + ( eta * Delta3[i] * Y1 ) );
        Weight[5] = Weight[5] + ( ( WeightChange[5] * alpha ) + ( eta * Delta3[i] * Y2 ) );

        Weight[6] = Weight[6] + ( ( WeightChange[6] * alpha ) + ( eta * Delta1[i] ) );
        Weight[7] = Weight[7] + ( ( WeightChange[7] * alpha ) + ( eta * Delta2[i] ) );
        Weight[8] = Weight[8] + ( ( WeightChange[8] * alpha ) + ( eta * Delta3[i] ) );

        for( int k = 0; k < 9; k++ ) {
            WeightChange[k] = OldWeight[k] - Weight[k];
            OldWeight[k] = Weight[k];
        }

        //Global Error
    for( int j = 0; j < 4; j++ ) {
        Y1 = ( InputOne[j] * Weight[0] ) + ( InputTwo[j] * Weight[1] ) + Weight[6];
        Y1 = 1 / ( 1 + Math.exp( -Y1 ) );

        Y2 = ( InputOne[j] * Weight[2] ) + ( InputTwo[j] * Weight[3] ) + Weight[7];
        Y2 = 1 / ( 1 + Math.exp( -Y2 ) );

        Y3 = ( Y1 * Weight[4] ) + ( Y2 * Weight[5] ) + Weight[8];
        Y3 = 1 / ( 1 + Math.exp( -Y3 ) );

        //System.out.println( Y3 + " " + Math.abs( Result[j] - Y3 ) );

        SumError = SumError + Math.pow( ( Result[j] - Y3 ) , 2 );
    }
        SumError = SumError * 0.5;
    }
    Count = Count + 1;  
        }

Where InputOne, InputTwo and Result are the truth table entries for XOR, the weights are randomly assigned and WeightChange is the momentum.

I then feed in the truth table again and every output is more or less the same as the last input it trained on.

Does anybody have any ideas?

Answer 1

You should train for case1(once),case2(once),case3(once),case4(once) --> repeat again until it learns all four. Not a single case. Make single iteration per case. You need to make it learn plastic. For your teaching scheme, when you teach case2, it forgets case1. You need to feed all cases repeatedly in a common while loop until total error diminishes to a lower limit.

When you make it learn only single case, it learns it good but forgets others. So you feed one after another and take a group of cases' total error (maybe sum of squared errors) to be smaller than a tolerance.