Getting value of 2 in normalized confusion matrix
-
12-12-2020 - |
Question
I have created confusion matrix with python and I have gotten weird results that made me worried:
as you can see, I have value of two, and if I understand correct, normalized matrix cannot have value higher than 1. and also, some columns' sum is not 1.
this is how I created this matrix:
df_confusion = pd.crosstab(df['labels'], df['prediction'])
df_conf_norm = df_confusion / df_confusion.sum(axis=1)
and then I just plot it.
my question is - is it possible to get value of 2? does the sum of column make sense not to be 1? do I have mistake? and if correct, what can explain that?
Solution
Switch the axes:
# Build the table
confusion_table = pd.crosstab(df["prediction"], df["labels"])
# Normalise
confusion_table = (confusion_table / consufion_table.sum(0))
# Verify
print(confusion_table.sum())
Alternatively, you can play with confusion_table.divide
and the axis
argument: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.divide.html
Licensed under: CC-BY-SA with attribution
Not affiliated with datascience.stackexchange