ANN Training Set Scaling

Question

I am working on an ANN using Backpropagation at the moment, for a classification task. I am however a little confused about normalizing the data set I am using(I do not have a strong stats/probability background).

A sample of the data is shown below:

5.1, 3.5, 1.4, 0.2, 1
5.2, 2.7, 3.9, 1.4, 2
5.9, 3.0, 5.1, 1.8, 3

Where the last token of each is the class.

Now, as I am using the Sigmoid transfer function, my network obviously cannot output a value greater than 1, so the data needs to be normalized/scaled.

My first question; would I need to scale both the features and the class, or just the class?

My second question, is there any 'de-facto' or commonly used method of doing such scaling?

Regards, Jack Hunt

Answer 1

It's usually recommended to also scale the features. Scaling should be as straightforward as scaled_feature = (feature - min(featurearray))/(max(featurearray) - min(featurearray)).

So for the first attribute column, the new data would be: (5.1-5.1)/(5.9-5.1); (5.2-5.1)/(5.9-5.1); (5.9-5.1)/(5.9-5.1)

Answer 2

Actually, the topic of scaling is one of the most important aspects in employing machine learning algorithms, especially ANNs.

Yes, the (feature-min / max - min) approach is one possibility.

However, there are a range of other methods. For example, you could use z-scores (in terms of standard deviations from the mean). Alternatively, there is also a technique called z-axis normalization, in which an extra component is added to a normalized vector. And sometimes, absolutely no scaling is required.