How to predict multiple values with Encog SVM in Java

Question

I've been working on a prediction system in Backpropagation Neural Networks with the library Encog long ago and now I've been researching the field of Support Vector Machines and I see that it is a lot more efficient than NN, my objective is to test how predictions behave in SVM and compare it with BPN.

I have two bidimensional arrays, one array is the INPUT and one is the IDEAL. Every row of the array is composed of 3 elements (real numbers between 1/9 and 9). There are 800 rows in the INPUT with the correspondent IDEAL. And there are another 2 arrays for the TEST set with 200 rows.

With BPN there is no problem, I train the network with 3 neurons in the Input Layer, 50 neurons in the Middle Layer and 3 neurons in the Output layer (converges in 5 min with error less than 0.01) and the Tests work fine.

In SVM, on the other hand I have a bit of a problem. I train the network, converges in 3 seconds and it seems to work fine, but when I test the data it predicts only 1 element in the output per row, not 3 asi in BPN.
I don't know what I'm doing wrong, I copy a simplified code and the output of the run.

JAVA CODE

/* SVM Structure */
SVM svm = new SVM(3, true); 

/* Training Set */
BasicMLDataSet trainingSet = new BasicMLDataSet(INPUT_ARRAY, IDEAL_ARRAY);

/* Train SVM */
SVMSearchTrain train = new SVMSearchTrain(svm, trainingSet);

int epoch = 1;
do {
   train.iteration();
   System.out.println("Epoch #" + epoch + " Error:" + train.getError());
   epoch++;
} while(train.getError() > 0.01);

// Test the SVM
BasicMLDataSet testSet = new BasicMLDataSet(INPUT_TEST_ARRAY, IDEAL_TEST_ARRAY);
//Normalize/Denormalize class -> 0.11111111111111111 - 9 into 0 - 1
NormalizedField norm = new NormalizedField(NormalizationAction.Normalize, null,9,0.11111111111111111,1,0);

for(MLDataPair dataPair: testSet ) {
   final MLData output = svm.compute(dataPair.getInput());
   System.out.print("Input: [ ");           
   int elementos = (orden*(orden-1))/2;
   for(int i=0; i<elementos; i++){
       System.out.print(norm.deNormalize(dataPair.getInput().getData(i))+" ");
   }
   System.out.print("] - ");
   System.out.print("Ideal: [ ");           
   for(int i=0; i<elementos; i++){
       System.out.print(norm.deNormalize(dataPair.getIdeal().getData(i))+" ");
   }
   System.out.print("] - ");
   System.out.print("Ideal: [ "); 
   for(int i=0; i<output.size();i++){
      System.out.print(norm.deNormalize(output.getData(0))+" ");
   }
   System.out.print("]\n");         
}

Encog.getInstance().shutdown();

OUTPUT

Epoch #1 Error:0.0040788759553631255
Input: [ 0.1111111111111111 0.14285714285714285 4.0 ] - Ideal: [ 0.1111111111111111 0.16666666666666666 4.0 ] - Actual: [ -0.41164113104867955 ]
Input: [ 0.125 0.125 4.0 ] - Ideal: [ 0.125 0.5 3.0 ] - Actual: [ -0.4085939985232357 ]
Input: [ 1.0 1.0 0.25 ] - Ideal: [ 1.0 0.5 0.25 ] - Actual: [ 1.6604401446928032 ]
.
.
.
Input: [ 8.0 4.0 0.16666666666666666 ] - Ideal: [ 8.0 4.0 0.2 ] - Actual: [ 7.545661375461256 ]

Any help will be greatly appreciated, thanks for reading!

Answer 1

I assume this is a regression problem, where you a number out of the SVM, and not a classification.

Neural networks are somewhat unique among models in that they can have multiple outputs. A SVM, when used for regression, has a single output. You can use a SVM to classify to multiple classes, however, for regression, you are limited to a single output.

If there are two different numbers that you are trying to predict, then you will need to use two SVM's. If you want to classify into multiple classes, there is more info here: http://www.heatonresearch.com/content/encog_svm_multiclass.html