The correctness of neural networks

Question

I have asked other AI folk this question, but I haven't really been given an answer that satisfied me.

For anyone else that has programmed an artificial neural network before, how do you test for its correctness?

I guess, another way to put it is, how does one debug the code behind a neural network?

Answer 1

Correctness is a funny concept in most of "soft computing." The best I can tell you is: "a neural network is correct when it consistently satisfies the parameters of it's design." You do this by training it with data, and then verifying with other data, and having a feedback loop in the middle which lets you know if the neural network is functioning appropriately.

This is of-course the case only for neural networks that are large enough where a direct proof of correctness is not possible. It is possible to prove that a neural network is correct through analysis if you are attempting to build a neural network that learns XOR or something similar, but for that class of problem an aNN is seldom necessary.

Answer 2

With neural networks, generally what is happening is you are taking an untrained neural network, and you are training it up using a given set of data, so that it responds in the way you expect. Here's the deal; usually, you're training it up to a certain confidence level for your inputs. Generally (and again, this is just generally; your mileage may vary), you cannot get neural networks to always provide the right answer; rather, you are getting the estimation of the right answer, to within a confidence range. You know that confidence range by how you have trained the network.

The question arises as to why you would want to use neural networks if you cannot be certain that the conclusion they come to is verifiably correct; the answer is that neural networks can arrive at high-confidence answers for certain classes of problems (specifically, NP-Complete problems) in linear time, whereas verifiably correct solutions of NP-Complete problems can only be arrived at in polynomial time. In layman's terms, neural networks can "solve" problems that normal computation can't; but you can only be a certain percentage confident that you have the right answer. You can determine that confidence by the training regimen, and can usually make sure that you will have at least 99.9% confidence.

Answer 3

You're opening up a bigger can of worms here than you might expect.

NN's are perhaps best thought of as universal function approximators, by the way, which may help you in thinking about this stuff.

Anyway, there is nothing special about NN's in terms of your question, the problem applies to any sort of learning algorithm.

The confidence you have in the results it is giving is going to rely on both the quantity and the quality (often harder to determine) of the training data that you have.

If you're really interested in this stuff, you may want to read up a bit on the problems of overtraining, and ensemble methods (bagging, boosting, etc.).

The real problem is that you usually aren't actually interested in the "correctness" (cf quality) of an answer on a given input that you've already seen, rather you care about predicting the quality of answer on an input you haven't seen yet. This is a much more difficult problem. Typical approaches then, involve "holding back" some of your training data (i.e. the stuff you know the "correct" answer for) and testing your trained system against that. It gets subtle though, when you start considering that you may not have enough data, or it may be biased, etc. So there are many researchers who basically spend all of their time thinking about these sort of issues!

Answer 4

I've worked on projects where there is test data as well as training data, so you know the expected outputs for a set of inputs the NN hasn't seen.

One common way of analysing the result of any classifier is use of an ROC curve; an introduction to the statistics of classifiers and ROC curves can be found at Interpreting Diagnostic Tests

Answer 5

I'm a complete amateur in this field, but don't you use a pre-determined set of data you know is correct?

Answer 6

I don't believe there is a single correct answer but there are well-proven probabilistic or statistical methods that can provide reassurance. The statistical methods are usually referred to as Resampling.

One method that I can recommend is the Jackknife.

Answer 7

My teacher always said his rule of thumb was to train the NN with 80% of your data and validate it with the other 20%. And, of course, make sure that data set is as comprehensive as you need.

Answer 8

If you want to find out whether the backpropagation of the network is correct, there is an easy way.

Since you calculate the derivate of the error landscape, you can check whether your implementation is correct numerically. You will calculate the derivative of the error with respect to a specific weight, ∂E/∂w. You can show that

∂E/∂w = (E(w + e) - E(w - e)) / (2 * e) + O(e^2).

(Bishop, Machine Learning and Pattern Recognition, p. 246)

Essentially, you evaluate the error to the left of the weight, evaluate it to the right of the weight and chheck if the numerical gradient is the same as your analytical gradient.

(Here's an implementation: http://github.com/bayerj/arac/raw/9f5b225d6293974f8adfc5f20dfc6439cc1bed35/src/cpp/utilities/utilities.cpp)

Answer 9

To me probably there is only one value(s) takes extra effort to verify, the gradient of the back propagation. I think Bayer's answer is actually commonly used and suggested. You need to write extra code to this but all are forward propagation matrix multiplications which is easy to write and verify.

There are some other issues which will prevent you from getting the best answer, for example:

• The cost function of NN is not concave so your gradient descent is not guaranteed to find the global optimum.
• Over/under fitting
• Not choosing the "right" features/model
• etc

However I think they are beyond the scope of programming bug.