Verification of trained system

Question

I have trained a system in order to detect some features from a set of scenarios. Now the system can detect and classify that set. How can I validate how that system works in real world? What mathematics tools should I use?

Answer 1

Did you also validate the data set with a separate test set so you can see how it performs with known data?

This answer has a good run down on how to do that: https://stats.stackexchange.com/questions/19048/what-is-the-difference-between-test-set-and-validation-set

It also is a best practice to ensure that you haven't overfit the data. The easiest way to do that is check the accuracy against the data that you used to train the data. If its not substantially better than the test data (this data should never be used to train a model) then you have not over fit.

After that, if you eventually find out the correct classification in your system, I would compare the accuracy there to the accuracy you experienced in the training and test environments to determine if its performing as intended.