Calculating AUC ratio in R

Question

I am generating ecological niche models for a set of species and I would like to use AUC as a metric for ecological niche quality. Steven Phillips, who developed Maxent, provides code in his Maxent manual for calculating the AUC in R. However, I am reading papers that report partial AUC ratios as a more robust and conceptually sound metric. I think I understand how to calculate partial AUC using the ROCR R package, but how does one calculate AUC ratio?

Here is the tutorial script from Phillips:

presence<-read.csv("bradypus_variegatus_samplePredictions.csv")
background<-read.csv("bradypus_variegatus_backgroundPredictions.csv")
pp<-presence$Logistic.prediction
testpp<-pp[presence$Test.or.train=="test"]
trainpp<-pp[presence$Test.or.train=="train"]
bb<-background$logistic

combined<-c(testpp,bb)
label<-c(rep(1,length(testpp)),rep(0,length(bb)))
pred<-prediction(combined,label)
perf<-performance(pred,"tpr","fpr")
plot(perf,colorize=TRUE)
performance(pred,"auc")@y.values[[1]] #RETURNS AUC

AUC<-function(p,ind){
    pres<-p[ind]
    combined<-c(pres,bb)
    label<-c(rep(1,length(pres)),rep(0,length(bb)))
    predic<-prediction(combined,label)
    return(performance(predic,'auc')@y.values[[1]])
}

b1<-boot(testpp,AUC,100) #RETURNS AUC WITH STANDARD ERROR
b1

Any advice or suggestions would be greatly appreciated! Thank you.

Answer 1

Without knowing the specifics of your dataset and application,

• Partial AUC: The area under only a portion of the curve. (usually picked because it is more robust or otherwise desirable, like you said)
• AUC ratio: The ratio of one AUC to another. (usually a reference of some sort)

Soo...

• Partial AUC ratio: The ratio of one partial AUC to another.

Answer 2

Package ROCR can calculate partial AUC values using the fpr.stop= parameter. As John said the ratio is just this value divided by the same calculation for your reference model.