How to correctly export Weight and Bias value of Backpropagation neural network into another programming language (Java)

Question

I created backpropagation Neural Network using Matlab. I tried to implement XOR gate using Matlab, then getting its weight and bias to create neural network in java. Network consist of 2 input neuron, 2 hidden layer each using 2 neuron and 1 output neuron. After train network, i got following weight and bias :

clear;
clc;
i = [0 0 1 1; 0 1 0 1];
o = [0 1 1 0];
net = newff(i,o,{2,2},{'tansig','logsig','purelin'});
net.IW{1,1} = [
    -5.5187   -5.4490;
     3.7332    2.7697
];
net.LW{2,1} = [
   -2.8093   -3.0692;
   -1.6685    6.7527
];
net.LW{3,2} = [
    -4.9318   -0.9651
];
net.b{1,1} = [
    2.1369;
    2.6529
];
net.b{2,1} = [
    -0.2274;
    -4.9512
];
net.b{3,1} = [
    1.4848
];

input  = net.IW{1,1};
layer  = net.LW{2,1};
output = net.LW{3,2};

biasinput = net.b{1,1};
biaslayer = net.b{2,1};
biasoutput= net.b{3,1};


a = sim(net,i);
a;

I simulate it using 1 and 1 as input got following result :

>> f = [1;1]

f =

     1
     1

>> sim(net,f)

ans =

   -0.1639

Then I tried to make simple java code to count this neural network. My code :

public class Xor {

    //Value of neuron
    static double[] neuroninput    = new double[2];
    static double[] neuronhidden1  = new double[2];
    static double[] neuronhidden2  = new double[2];
    static double[] neuronoutput   = new double[2];

    //Weight variable init
    //For first hidden layer
    static double[] weighthidden11 = new double[2];
    static double[] weighthidden12 = new double[2];

    //for second hidden layer
    static double[] weighthidden21 = new double[2];
    static double[] weighthidden22 = new double[2];

    //for output layer
    static double[] weightoutput   = new double[2];
    //End of weight variable init

    //Bias value input
    static double[] biashidden1    = new double[2];
    static double[] biashidden2    = new double[2];
    static double[] biasoutput     = new double[1];

    public static void main(String[] args) {
        neuroninput[0] = 1;
        neuroninput[1] = 1;

        weighthidden11[0] = -5.5187;
        weighthidden11[1] = -5.4490;
        weighthidden12[0] =  3.7332;
        weighthidden12[1] =  2.7697;

        weighthidden21[0] = -2.8093;
        weighthidden21[1] = -3.0692;
        weighthidden22[0] = -1.6685;
        weighthidden22[1] =  6.7527;

        weightoutput[0]    = -4.9318;
        weightoutput[1]    = -0.9651;

        biashidden1[0] = 2.1369;
        biashidden1[1] = 2.6529;

        biashidden2[0] = -0.2274;
        biashidden2[1] = -4.9512;

        biasoutput[0]  = 1.4848;

        //Counting each neuron (Feed forward)
        neuronhidden1[0] = sigma(neuroninput,weighthidden11,biashidden1[0]);
        neuronhidden1[0] = tansig(neuronhidden1[0]);

        neuronhidden1[1] = sigma(neuroninput,weighthidden12,biashidden1[1]);
        neuronhidden1[1] = tansig(neuronhidden1[1]);


        neuronhidden2[0] = sigma(neuronhidden1,weighthidden21,biashidden2[0]);
        neuronhidden2[0] = logsig(neuronhidden2[0]);

        neuronhidden2[1] = sigma(neuronhidden1,weighthidden22,biashidden2[1]);
        neuronhidden2[1] = logsig(neuronhidden2[1]);

        neuronoutput[0] = sigma(neuronhidden2,weightoutput,biasoutput[0]);
        neuronoutput[0] = purelin(neuronoutput[0]);
        System.out.println(neuronoutput[0]);
    }

    static double tansig(double x) {
        double value = 0;
        value = (Math.exp(x) - Math.exp(-x)) / (Math.exp(x) + Math.exp(-x));
        return value;
    }

    static double logsig(double x) {
        double value = 0;
        value = 1 / (1+Math.exp(-x));
        return value;
    }

    static double purelin(double x) {
        double value = x;
        return value;
    }

    static double sigma(double[] val, double[] weight, double hidden) {
        double value = 0;
        for (int i = 0; i < val.length; i++) {
            value += (val[i] * weight[i]);
            //System.out.println(val[i]);
        }
        value += hidden;
        return value;
    }
}

But it got result as following :

-1.3278721528152158

My question, is there any error or my mistake in exporting weight and bias value from matlab to java? Maybe I made mistake in my java program? Thank you verymuch..

Answer 1

I think the problem is the normalization: http://www.mathworks.com/matlabcentral/answers/14590

If you work with 0,1 inputs, you have to use the f(x)=2*x-1 normalization function, which transforms the values to the [-1; 1] interval, then g(x)=(x+1)/2 to transform back the output to [0; 1]. Pseudocode:

g( java_net( f(x), f(y) ) ) = matlab_net(x, y)

I tried this with an other network and worked for me.

Answer 2

Your problem is most surely related to your JAVA version of the Matlab sim() command.

It is a complex Matlab command with many settings affecting the architecture of the network to be simulated. To make debugging easier, try and implement the sim() command yourself in Matlab. Possibly reduce the number of layers until you have a match in Matlab between sim()-builtin and your own sim version. When that is working convert to JAVA.

EDIT:

The Reason for re-implementing the sim() function in Matlab, is that if you can't implement it here, you won't be able to properly implement it in JAVA either. Feed forward networks are quiet easy to implement using Matlab vector notation.