Attempting to Implement a Neural Network to solve XOR

Question

I'm hoping some of the more experienced users here might have some suggestions for me. I am implementing a neural network with 2 inputs, 2 hidden nodes, and 1 output.

I have used the sigmoid activation function on both the hidden layer and the output and I'm using back propagation. I am fairly certain I understand the theory correctly. I have the program calculating gradients, updating weights and biases, and I use momentum and strength variables for adjusting.

The point of using multiple layers is to solve non-linearly separable problems, but I have only been able to solve the linear seperable AND and OR boolean functions so far. I have tried playing with all kinds of different momentum and strength settings to no avail.

My usual outputs are always exactly the same for all 4 variables. It was near 0.55 for awhile, until I played with settings and now they're all outputting 0.9. If I remove the bias the first value goes to zero, but not the fourth.

Any suggestions?

Answer 1

To answer my own question ..

After a lot of trial and error, I threw caution to the wind and tried using tanh(x) instead of sigmoid .. and after only a little tweeking, it WORKS!

If anyone else has been struggling with one of these nets, it might work for you.

The derivative is (1 - tanh(x))(1 + tanh(x)).