ROC curve for discrete predictions

Question

I have a classifier that predicts the probability and class for a binary-class dataset. I want to compare it with another predictor which outputs only the binary class for each case. I obtained the performance scores but now I need to show the difference in ROC curve. I can plot ROC curve for continuous predictions (probabilities) using packages like ROCR and pROC in R. As the second predictor outputs only binary class, I am wandering around looking for something which I can use to plot ROC curve for discrete predictions (binary). I found some script plotroc.py in python but I cant find anyting similar in R. Can anyone point me to some R package or other reliable package that I can use to plot ROC curve for my data?

Thanks!

Answer 1

I would rather binarize the predictions of your first predictor (using a ROC curve to choose the threshold) and compare them with a Fisher's Exact Test or something like this. It doesn't make sense to build a ROC curve for a binary prediction.