One vs rest multiclass classification using LIBSVM. matlab

Question

I am trying to implement one vs rest multiclass classification using LIBSVM.

This link was useful http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/ovr_multiclass/ but i get an error in the function 'ovrpredict()'.

The function is as below:

function [pred, ac, decv] = ovrpredict(y, x, model)

labelSet = model.labelSet;
labelSetSize = length(labelSet);
models = model.models;
decv= zeros(size(y, 1), labelSetSize);

for i=1:labelSetSize
  [l,a,d] = svmpredict(double(y == labelSet(i)), x, models{i});
  decv(:, i) = d * (2 * models{i}.Label(1) - 1);             % ERROR IN THIS LINE
end
[tmp,pred] = max(decv, [], 2);
pred = labelSet(pred);
ac = sum(y==pred) / size(x, 1);

The error message i get is Reference to non-existent field 'Label'.

Any suggestions will be really helpful.

---

EDIT 1

The code used to call the functions:\

[trainY trainX]=libsvmread('libfacecombine.train');
[testY testX]=libsvmread('libfacetest.train');
model=ovrtrain(trainY,trainX,'-c 8 -g 4');
[~,accuracy,~]=ovrpredict(testY,testX,model);

The training and testing data viz 'libfacecombine.train' and 'libfacetest.train' is written obtained from .csv files:

f1=createdabase(f);     % where createdatabase is a function to read various images from a folder and arrange into 1D array
[sig1 mn1]=pcam(f1);    % where pcam is a function to find 'pca'(sig1) and 'mean'(mn1) of the data

%labelling is done this way:
%Positive class
        label=[];
        for i=1:length(sig1)
            for j=1:1
                label(i,j)=+1;
            end
        end
    csvwrite('face1.csv',[label sig1]);

%Negative class
        label1=[];
            for i=1:length(sig2)             % sig2 obtained in same way as sig1
                for j=1:1
                    label1(i,j)=-1;
                end
            end
        csvwrite('face2.csv',[label sig2]);

Using "append" mode both these files are joined and converted to .train files. Same thing is done for the testing data.

EDIT 2

I have 5 classes. And labeling is done as: Class 1: +1 contains features from 4 images of Face 1 and -1 contains features from 4 images of not Face 1(faces 2,3,4 and 5). Class 2: +2 contains features from 4 images of Face 2 and -2 contains features from 4 images of not Face 2(faces 1,3,4 and 5).... Class 5: +5 contains features from 4 images of Face 5 and -5 contains features from 4 images of not Face 5(faces 1,2,3 and 4). All these features along with the labels are written in the above given order to .csv files and then converted to .train format. Hence i obtain the training file.

For the test image i take one image of face 1 and give its true label i.e +1 and written into .csv file and then converted to .train. Hence i obtain the testing file. When i run the program i obtain results such as:

Accuracy=92%(12/13)classification; 
Accuracy=61%(8/13)classification;
Accuracy=100%(13/13)classification;
Accuracy=100‌​%(13/13)classification;
Accuracy=100%(13/13)classification;
Accuracy=100%(13/13)cla‌​ssification; 

Why am i obtaining 6 accuracy values when i have only 5 classes?

Answer 1

Finding the solution to something like this is tricky over the internet, but let's have a try. This post is comprised of questions rather than answers. However, I believe that if you answer them all you will find your bug without further help – or at least be 90% of the way there.

All these steps are sensible for debugging any type of MATLAB program.

---

Ensuring a clear workspace

Old versions of variables hanging around in the workspace can make it hard to debug. A mispelling in a variable name can cause an old version to be used by accident. Clear your workspace with clear at the start of your debugging.

---

Running a simpler program

I have compiled libsvm, added the ovr_multiclass addons, and I can succesfully run the following example script that I made up:

clear

% random train and test data
trainX = rand(10, 4);
trainY = randi(4, 10, 1);
testX = rand(10, 4);
testY = randi(4, 10, 1);

model=ovrtrain(trainY,trainX,'-c 8 -g 4'); 
[~,accuracy,~]=ovrpredict(testY,testX,model);

Can you run this, or do you get the same error as before? This sort of minimum working example is really useful for debugging. Using small, user-generated data ensures that the errors don't come from unexpected sources and helps to narrow down the cause.

---

Examining the models cell array

You state that the error is occuring in this :

decv(:, i) = d * (2 * models{i}.Label(1) - 1);             % ERROR IN THIS LINE

The critical part of the line is models{i}.Label(1). This is taking a cell array models, and extracting the ith item. This ith item is expected to be a structure, with a field called Label. Label is expected to be a non-empty array, from which the first element can be extracted. The models cell array is a field in the structure model, which is passed in as a third argument to ovrpredict.

After running my very simple test script above, I run the following diagnostics in the MATLAB command window:

>> models = model.models

models = 

    [1x1 struct]
    [1x1 struct]
    [1x1 struct]
    [1x1 struct]

>> models{1}

ans = 

    Parameters: [5x1 double]
      nr_class: 2
       totalSV: 6
           rho: -1.2122
         Label: [2x1 double]
    sv_indices: [6x1 double]
         ProbA: []
         ProbB: []
           nSV: [2x1 double]
       sv_coef: [6x1 double]
           SVs: [6x4 double]

>> models{1}.Label

ans =

     0
     1

If you do the same, do you get the same result? If not, post your output to these commands in an edit to your question.

---

Debugging in the function

If the model looks OK but you are still getting the error, turn on the error debugger by typing this into the command window:

dbstop if error

When MATLAB encounters an error in a function it will now pause and allow you to inspect all the variables.

Run your program again (or mine, if you are getting the error with the minimum working example I posted). The program should pause when the error occurs. Your command prompt should change from >> to K>>.

Run through the steps above to example the model cells array. Then try copying, pasting and running the line of code which gave the error in the command window. Try running a smaller portion of it, e.g. models{i}.Label(1) then 2 * models{i}.Label(1) - 1.

Type dbquit to exit the debugger, and dbclear if error to turn off the automatic debugging on error.

(See also my question below about the error message – make sure that the error is actually occuring where you think it is!)

---

Edit: Some extra questions

• What version of MATLAB are you using? e.g. R2013a

• If you type which ovrpredict into the MATLAB command line, do you see the path to the file that you expect? (i.e. the correct path to ovrpredict.m as you have it saved on your computer)

• Does your ovrpredict.m file (as pointed to by which ovrpredict) contain exactly what you have pasted in your question? The error you are getting suggests their might be a tiny difference, e.g. an extra space.

Answer 2

Though I am not sure on how to solve this, here are my observations:

In ovrtrain a model is created with the only addition field being labelSet.

Later in overpredict the code tries to read out label.

So, I see three options:

1. The creation of the model is wrong
2. A step in the middle is forgotten
3. The using of the model is wrong

EDIT: THIS PART IS PROBABLY NOT THE SOLUTION, AS MENTIONED IN THE COMMENTS

I have insufficient information to know which of these applies, but if no examples are provided with the code you could try replacing Label by labelSet everywhere.

Note that if it is point 1 or 3, this code should always provide an error and never run properly.