Can a perceptron forget?

Question

I would like to build an online web-based machine learning system, where users can continuously add classified samples, and have the model updated online. I would like to use a perceptron or a similar online-learning algorithm.

But, users may make mistakes and insert irrelevant examples. In that case, I would like to have the option to delete a specific example, without re-training the perceptron on the entire set of examples (which may be very large).

Is this possible?

Answer 1

As I understand the process, altering a perceptron without retraining is impossible. The weight adjustments are not only relative to that specific example but also relative to the other training examples that have gone before. Identifying the incorrectly classified instance and removing it from the test set before retraining the model would seem to be the most effective way of correcting the weights.

I think it's worth pointing out that in comparison to other machine learning algorithms, perceptrons are relatively resistent to noise and incorrectly classified instances in the training set. If you're encountering a large number of misclassified instances, it would seem more prudent to have better validation at the point you ingest the data prior to training than to come up with some way to correct for misclassified instances after the perceptron has been trained. If that's not possible and you're able to identify the incorrectly classified instances as such, then removing them and retraining would seem the only way to effectively remove the impact of the misclassified instances.