create ROC from 10 different thresholds

Question

I have output from svmlight which has x=predictions (0.1,-0.6,1.2, -0.7...), y=actual class {+1,-1}. I want to create an ROC curve for 10 specific different thresholds (let t be a vector that contains 10 different threshold values). I checked ROCR package but I didn't see any option for supplying threshold vector. I need to calculate TPR and FPR for each threshold value and plot. Is there any other way to do that ? I am new to R programming.

Answer 1

ROCR creates an ROC curve by plotting the TPR and FPR for many different thresholds. This can be done with just one set of predictions and labels because if an observation is classified as positive for one threshold, it will also be classified as positive at a lower threshold. I found this paper to be helpful in explaining ROC curves in more detail.

You can create the plot as follows in ROCR where x is the vector of predictions, and y is the vector of class labels:

pred <- prediction(x,y) 
perf <- performance(pred,"tpr","fpr")
plot(perf)

If you want to access the TPR and FPR associated with all the thresholds, you can examine the performance object 'perf':

str(perf)

The following answer shows how to obtain the threshold values in more detail:

https://stackoverflow.com/a/16347508/786220

Answer 2

You can do that with the pROC package. First create the ROC curve (for all thresholds):

myROC <- roc(y, x) # with the x and y you defined in your question

And then you query this curve for the 10 (or any number of) thresholds that you stored in t:

coords(myROC, x = t, input="threshold", ret = c("threshold", "se", "1-sp"))

Sensitivity is your TPR while 1-Specificity is your FPR.

---

Disclaimer: I am the author of pROC.