Is there a way to set up a multi-hidden layer neural network with the mlp method in the caret package?

Question

The mlp method in package caret calls the mlp function in RSNNS. In the RSNNS package, I can set up as many hidden layers in the neural net as I like by setting the size parameter, e.g.

data(iris)

#shuffle the vector
iris <- iris[sample(1:nrow(iris),length(1:nrow(iris))),1:ncol(iris)]

irisValues <- iris[,1:4]
irisTargets <- decodeClassLabels(iris[,5])
#irisTargets <- decodeClassLabels(iris[,5], valTrue=0.9, valFalse=0.1)

iris <- splitForTrainingAndTest(irisValues, irisTargets, ratio=0.15)
iris <- normTrainingAndTestSet(iris)

model <- mlp(iris$inputsTrain, iris$targetsTrain, size=c(5,7), learnFuncParams=c(0.1), 
             maxit=50, inputsTest=iris$inputsTest, targetsTest=iris$targetsTest)

Will set up a neural net with two hidden layers of 5 and 7 nodes respectively. I want to use the caret package because it has functionality for doing parameter/model searches, as well as parallel implementations for a cluster. In caret, when I look up the method, it can only be tuned with one parameter, size, e.g.

data(iris)

mlpGrid <- data.frame(.size=3)
model2<-caret::train(Species~. , iris, method='mlp', tuneGrid=mlpGrid)

Sets up a neural net with a 3-node single hidden layer.

I've tried adding other columns to mlpGrid and such, but caret doesn't seem to allow for adding a second (or more) hidden layer.

Answer 1

You should use caret's "mlpML" method insted of "mlp". It do uses mlp function from RSNNS, but you are able to define the number of neurons per hidden layer separately. For instance, the following code should do the work. You define your customized grid with the definition of your layers, each layer (1, 2, and 3) and how many neurons per layer.

mlp_grid = expand.grid(layer1 = 10,
                       layer2 = 10,
                       layer3 = 10)

mlp_fit = caret::train(x = train_x, 
                y = train_y, 
                method = "mlpML", 
                preProc =  c('center', 'scale', 'knnImpute', 'pca'),
                trControl = trainControl(method = "cv", verboseIter = TRUE, returnData = FALSE),
                tuneGrid = mlp_grid)

Given the verboseIter=TRUE it shows that the values were indeed applied

+ Fold01: layer1=10, layer2=10, layer3=10 
+ Fold02: layer1=10, layer2=10, layer3=10 
+ Fold03: layer1=10, layer2=10, layer3=10
...

Answer 2

The short answer is that I don't believe Caret supports multi-hidden layer networks using the mlp method.

If you are set on using the mlp (Or need it to call RSNSS), then you might consider chaining the networks together, as in feeding the output of the first into the inputs of the second, simulating another hidden layer. This would obviously be more difficult to implement, and lose many of the optimizations that make you want to use Caret in the first place.

On the other hand, Caret offers an alternative which uses the neuralnet package (method = 'neuralnet'). This would allow you to use Caret with a multi-hidden layer neural network. However it looks from the documentation that only 3 hidden layers are supported.