Getting 'pred_table' information for predicted values of a model in 'statsmodels'

Question

In the Python package statsmodels, LogitResults.pred_table can be conveniently used to get a "confusion matrix", for arbitrary an arbitrary threshold t, for a Logit model of the form

mod_fit = sm.Logit.from_formula('Y ~ a + b + c', train).fit() 
...
mod_fit.pred_table(t) 
#Conceptually: pred_table(t, predicted=mod_fit.predict(train), observed=train.Y)

Is there a way to get the equivalent information for test data? For example, if I

pred = mod_fit.predict(test)

how do I get the equivalent of

mod_fit.pred_table(t, predicted=pred, observed=test.Y)

Is there a way to get statsmodels to do this (e.g. a way to build construct a LogitResults instance from pred and train.Y), or does it need to be done "by hand" — and if so how>

Answer 1

That's a good idea and easy to add. Can you post a github issue about it? You can do this with the following code

import numpy as np
pred = np.array(mod_fit.predict(test) > threshold, dtype=float)
table = np.histogram2d(test.Y, pred, bins=2)[0]

Answer 2

Here's another way, using bincount:

from __future__ import division
import numpy as np

def confusionmatrix( true, predicted, classnames="0 1", verbose=1 ):
    """ true aka y, observed class ids: ints [0 .. nclass) or bools
        predicted aka yhat: ints or bools, e.g. (probs > threshold)
    -> e.g.
        confusion matrix, true down, predicted across:
        [[0 2]  -- true 0, pred 0 1
         [7 1]  -- true 1, pred 0 1
    """
    true = np.asarray( true, dtype=int )
    pred = np.asarray( predicted, dtype=int )
    ntrue, npred = true.max() + 1, pred.max() + 1
    counts = np.bincount( npred * true + pred, minlength = ntrue * npred )  # 00 01 10 11
    confus = counts.reshape(( ntrue, npred ))
    if verbose:
        print "true counts %s:      %s" % (classnames, np.bincount(true))
        print "predicted counts %s: %s" % (classnames, np.bincount(pred))
        print "confusion matrix, true down, predicted across:\n", confus
    return confus

#...............................................................................
if __name__ == "__main__":
    n = 10
    np.random.seed( 7 )
    y = np.random.randint( 0, 2, n )
    p = np.random.randint( 0, 2, n )
    print "true:", y
    print "pred:", p
    confusionmatrix( y, p )