Denormalize values after prediction in R

Question

I am not sure whether I am denormalizing data correctly. I have one output variable and several input variables. I am normalizing them using the RSNNS package. Suppose x is an input matrix (NxM) where each of the N rows is an object with M features. And y is a vector (N) with corresponding answers.

nx <- normalizeData(x, type='0_1')

After that, some of the data are used to make the model and some for prediction. Suppose pred.ny are predicted values. These values are normalized.

pred.y <- denormalizeData(pred.ny, getNormParameters(nx))

Is this correct? How does it work? It's clear for one input it can use the min and max values previously used for normalization. But how does it work if each input was normalized separately, using its own min and max value?

update Here is a toy-example, where '0_1' looks better than 'norm'. 'norm' make huge training error and almost constant prediction.

x <- runif(1020, 1, 5000)
y <- sqrt(x)
nx <- normalizeData(x, type='0_1')
ny <- normalizeData(y, type='0_1')
model <- mlp(nx[1:1000], ny[1:1000], size = 1)
plotIterativeError(model)
npy <- predict(model, matrix(nx[1001:1020], ncol=1))
py <- denormalizeData(npy, getNormParameters(ny))
print(cbind(y[1001:1020], py))

Answer 1

There are two things going on here:

1. Training your model, i.e. setting the internal coefficients in the neural network. You use both the inputs and output for this.
2. Using the model, i.e. getting predictions with fixed internal coefficients.

For part 1, you've decided to normalize your data. So the neural net works on normalized data. So you have trained the neural net

• on the inputs f_X(X) rather than X, where f_X is the transform you used to the matrix of original inputs to produce the normalized inputs.
• on the outputs f_y(y) rather than y, where f_y is the transform you applied to the vector of outputs to get the normalized outputs.

In terms of your original inputs and outputs your trained machine now looks like this:

• Apply the normalization function f_X to inputs to get normalized inputs f_X(X).
• Run neural net with normalized inputs to produce normalized outputs f_y(y).
• Apply the denormalization function f_y^-1 to the normalized outputs f_y(y) to get y.

Note that f_X(X) and f_y, and hence f_y^-1, are defined on the training set.

So in R you might write something like this to get training data and normalize it, the first 100 rows

tx <- x[1:100,]
ntx <- normalizeData(tx, type='0_1')
ty <- y[1:100]
nty <- normalizeData(ty, type='0_1')

and something like this to denormalize the predicted results

pred.y <- denormalizeData(pred.ny, getNormParameters(nty))
                                                       # nty (or ny) not nx here

The slight concern I have is that I'd prefer to normalize the features used in prediction using the same transform f_X that I used for training, but looking at the RSNNS documentation this facility doesn't appear to be present (it would be easy to write it yourself, however). It would probably be OK to normalize the prediction features using the whole X matrix, i.e. including the training data. (I can also see it might be preferable to use the default, normalization to z-score that RSNNS provides rather than the "0_1" choice you have used.)