Multiple ROC curves in one plot ROCR

Question

Is it possible to plot the roc curve for diffrent classifiers in the same plot using the ROCR package? I've tried:

>plot(perf.neuralNet, colorize=TRUE)
>lines(perf.randomForest)

But I get:

Error en as.double(y) : cannot coerce type 'S4' to vector of type 'double'

Thank you!

Answer 1

The problem with your lines-approach is that there is no generic S4 lines function for an object of class performance defined in the ROCR package. But you can use the generic plot function as you did with an additional add = TRUE argument. For example this is partly from the example page of ?plot.performance:

library(ROCR)
data(ROCR.simple)
pred <- prediction( ROCR.simple$predictions, ROCR.simple$labels )
pred2 <- prediction(abs(ROCR.simple$predictions + 
                        rnorm(length(ROCR.simple$predictions), 0, 0.1)), 
        ROCR.simple$labels)
perf <- performance( pred, "tpr", "fpr" )
perf2 <- performance(pred2, "tpr", "fpr")
plot( perf, colorize = TRUE)
plot(perf2, add = TRUE, colorize = TRUE)

OR, you can store all your predictions in a matrix and do all the subsequent steps in one:

preds <- cbind(p1 = ROCR.simple$predictions, 
                p2 = abs(ROCR.simple$predictions + 
                rnorm(length(ROCR.simple$predictions), 0, 0.1)))

pred.mat <- prediction(preds, labels = matrix(ROCR.simple$labels, 
                nrow = length(ROCR.simple$labels), ncol = 2) )

perf.mat <- performance(pred.mat, "tpr", "fpr")
plot(perf.mat, colorize = TRUE)

Btw, if you for some reason really wanted to use lines to plot consecutive ROC curves you would have to do sth. like this:

plot(perf) 
lines(perf2@x.values[[1]], perf2@y.values[[1]], col = 2)

Answer 2

Echoing @adibender, and adding a comment: the example doesn't cover how to set separate colors for each individual curve using the second (plot all at once) approach. In this case, pass col as a list:

library(ROCR)
data(ROCR.hiv)
x   <- prediction(ROCR.hiv$hiv.nn$predictions, ROCR.hiv$hiv.nn$labels)
ROC <- performance(x, "tpr", "fpr")
plot(ROC, col = as.list(1:10))

Answer 3

R has functions for draw several plots in one window. And if package doesn't support several plots in one window,you can solve problem with standard instruments of R. Other way: Example of several ROCs Article with this script:An example of ROC curves plotting with ROCR