Error function in Artificial Neural Network trained using backpropogation

Question

On various literature I keep seeing reference of

error function

but I'm not quite sure what it means. I am using sigmoid function for activation. Does the error function mean the following equation:

differential = actualOutput(1-actualOutput)

error = (expectedOutput-actualOutput)(differential)

or is the following:

error = expectedOutput-actualOutput

?

Answer 1

Your answer

The error function is the function which you try to minimize. What you have listed above is a set of error functions, and the derivatives of some of them. It might be a bit confusing when litterature uses the same term when the minimizing function has been derivated. Just remember that we wish to minimize the error in our network, and the functions which helps us achieve it is the error function.

---

The error function

The most common error function is the the of the squares of the differences between all output_desired and output_actual.

The derivative of the error function for the output layer