混乱矩阵的信息标准

Question

可以使用 Akaike信息标准（AIC）, ，这说明了拟合的好处以及用于模型创建的参数数量。 AIC涉及计算该模型的似然函数的最大值（l）。一个人如何计算 l, ，给定分类模型的预测结果，表示为混淆矩阵？

Answer 1

Information-Based Evaluation Criterion for Classifier's Performance by Kononenko and Bratko is exactly what I was looking for:

Classification accuracy is usually used as a measure of classification performance. This measure is, however, known to have several defects. A fair evaluation criterion should exclude the influence of the class probabilities which may enable a completely uninformed classifier to trivially achieve high classification accuracy. In this paper a method for evaluating the information score of a classifier''s answers is proposed. It excludes the influence of prior probabilities, deals with various types of imperfect or probabilistic answers and can be used also for comparing the performance in different domains.

Answer 2

It is not possible to calculate the AIC from a confusion matrix since it doesn't contain any information about the likelihood. Depending on the model you are using it may be possible to calculate the likelihood or quasi-likelihood and hence the AIC or QIC.

What is the classification problem that you are working on, and what is your model?

In a classification context often other measures are used to do GoF testing. I'd recommend reading through The Elements of Statistical Learning by Hastie, Tibshirani and Friedman to get a good overview of this kind of methodology.

Hope this helps.