Is it possible to get an ROC curve using Relu activation?

Question

Based on my understanding, given that Relu doesn't provide probabilities unlike Softmax, it's not possible to plot an ROC curve. However, is there some way to convert the output from a Relu to something like an ROC curve. I'm interested in doing so because it's important for my project (image classification) to be able to adjust the classification thresholds.

Answer 1

YES

The typical way to do a ROC curve is to have probability outputs, set a threshold, and calculate the sensitivity and specificity.

y | prob
----------
0 | 0.2
0 | 0.3
1 | 0.7
1 | 0.8

At a threshold of 0, we classify everyting as $1$, so sensitivity of $1$ and specificity of $0$. At a threshold of $0.25$, we get $0,1,1,1$, so sensitivity of $1$ and specificity of $0.5$. Et cetera...

Plot those $(1, 1)$, $(0.5, 1)$ points to start building your ROC curve.

This idea of setting a threshold and calculating the sensitivity and specificity does not assume probability outputs, so let's do the same with non-probability outputs.

y | output
----------
0 | 2
0 | 3
1 | 7
1 | 8

At a threshold of 0, we classify everyting as $1$, so sensitivity of $1$ and specificity of $0$. At a threshold of $2.5$, we get $0,1,1,1$, so sensitivity of $1$ and specificity of $0.5$. Et cetera...