Number of Passes for Perceptron

Question

I am trying to implement Perceptron Algorithm but I am not able to figure out following points.

• what should be ideal value for iteration number
• is this algorithm suitable for large volumes of data?
• does threshold changes with iterations?
• if yes what difference does it make in final output?

Answer 1

The Perceptron is not a specific algorithm, it's a name of a cluster of algorithms. There're 2 major differences between these algorithms.

1. Integrate and fire rule

Let the input vector be x, the weights vector be w, the threshold be t and the output value be P(x). There're various function to calculate P(x):

1. binary: P(x) = 1 (if w * x>=t) or 0 (otherwise)
2. semi-linear: P(x) = w * x (if w * x>=t) or 0 (otherwise)
3. hard limit: P(x) = t(if w * x>=t) or w * x (if 0<w * x<t) or 0 (otherwise)
4. Sigmoid: P(x) = 1 / 1+e^(w * x)

and many others. So it's hard to say what difference does the threshold make in final ouptut, because it depends on which integrate and fire function you use.

2. Learning rule

The learning rule of the Perceptron is various too. The most simple and common one is

w -> w+ a * x* (D(x)- P(x))

where a is the size of a learning step, and D(x) is the expected output to x. So it's also hard to say that what should be a ideal value of iterations, because it depends on the value of a and how many training samples you do have.

Therefore, does thresold changes with iterations? it also depends. The simple and common learning rule above doesn't modify the threshold while training, but there're some other learning rules do modify it.

Btw, you also asked that is this algorothm suitable for large volume of data? The main metrics to measure the suitability of a classifier for certain dataset, is the linear separability of the dataset, not the scale of it. Keep in mind that the Single-layer Perceptron has very bad performance for the datasets which are not linear separable. The scale of the dataset does NOT that matter.