Classify data with a perceptron in MATLAB

Question

I am generating random data that can be separated linearly. I want to write my own version of a perceptron to separate them. I know there are some post, that have similar problems - but I can't find my mistake. I am really stuck. The algorithm works, but doesn't seem to converge. I would appreciate if you could help me.

My code:

single_layer_perceptron.m

% INPUT
% amount of values
points = 20;
% stepsize
s = 1.0;

% INITIALIZE
% Booleans
TRUE  = 1;
FALSE = 0;

% generate data
D = generateRandomData(points);
% x-values
x = D(:,1);
% y-values
y = D(:,2);
% training set
d = D(:,3);
% weights
w = zeros(3,1);
% bias
b = 1;
% sucsess flag
isCorrect = FALSE;
% correctly predicted values counter
p = 0;
% COMPUTE 

% while at east one point is not correctly classified
while isCorrect == FALSE
    % for every point in the dataset
    for i=1 : points
        % calculate outcome with current weight
        c = heaviside(b * w(1) + x(i) * w(2) + y(i) * w(3));
        % compare output with training set
        a = errorFunction(c,d(i)); 
        % if outcome was wrong
        if a ~= 0
            % ajust weights
            w(1) = w(1) + a*s*b;
            w(2) = w(2) + a*s*x(i);
            w(3) = w(3) + a*s*y(i);
        else
            % increase correctness counter
            p = p + 1;
        end
    end

    %disp(w);
    disp(p);

    if p >= points
       isCorrect = TRUE;
    end

    p = 0;

end

generateRandomData.m

function f = generateRandomData(points)
% generates random data that can be lineary seperated (silent)
% generate random function y = mx + n
m = 2  * rand * sign(randn);   % in (-2,2)/0
n = 10  * rand + 5;            % in (5,15)

% generate random points
x = 20 * rand(points,2);        % in ((0,20), (0,20))

% labeling
f = [x, zeros(points,1)];
for i=1:length(x(:,1))
    y = m*x(i,1) + n;
    if x(i,2) > y
        f(i,3) = 1;
    end    
end

end

activationFunctionHeaviside.m

function f = activationFunctionHeaviside(x)
f = (1/2)*(sign(x)+1);
end

errorFunction.m

function f = errorFunction(c,d)
% w has been classified as c - w should be d

if c < d 
    % reaction too small 
    f = -1;
elseif c > d
    % reaction too large
    f = 1;
else
    % reaction correct
    f = 0;
end

end

Thanks a lot!

Answer 1

The single-layer perceptron is a linear binary classifier that does not converge when the data is not linearly separable. If we plot the data, we get both classes are overlapping.

We can solve this use, by adding a tolerance to your function generateRandomData.m

function f = generateRandomData(points)
% generates random data that can be lineary seperated (silent)
% generate random function y = mx + n
m = 2  * rand * sign(randn);   % in (-2,2)/0
n = 10  * rand + 5;            % in (5,15)

% generate random points
x = 20 * rand(points,2);        % in ((0,20), (0,20))

% tolerance
tol = 0.5;

% labeling
f = [x, -ones(points,1)];
for ii=1:size(f,1)
    y = m*f(ii,1) + n;
    if f(ii,2) > y+tol
        f(ii,3) = 1;
    elseif f(ii,2) < y-tol
        f(ii,3) = 0;
    else
        f(ii,1) = f(ii,1)+2*tol;
        f(ii,3) = 1;
    end    
end
end

However, your code still does not converge because your errorFunction.m has switched signs. It should be like this:

function f = errorFunction(c,d)
% w has been classified as c - w should be d

if c < d 
    % reaction too small 
    f = +1;
elseif c > d
    % reaction too large
    f = -1;
else
    % reaction correct
    f = 0;
end

end

Once, we do these changes, we get a nice linear classification:

Code to plot the hypothesis:

% Plot
idx = logical(D(:,3));
Xax = 0:20; Yax=-(b*w(1)+Xax*w(2))/w(3);
figure;
hold on;
scatter(D(idx,1),D(idx,2),'bo')
scatter(D(~idx,1),D(~idx,2),'rx')
plot(Xax,Yax,'k--')