When do I have to use aucPR instead of auROC? (and vice versa)

Question

I'm wondering if sometimes, to validate a model, it's not better to use aucPR instead of aucROC? Do these cases only depend on the "domain & business understanding" ?

Especially, I'm thinking about the "unbalanced class problem" where, it seems more logical to use the aucPR because recall and precision are well-used metrics for this problem.

Answer 1

Yes, you are correct that the dominant difference between the area under the curve of a receiver operator characteristic curve (ROC-AUC) and the area under the curve of a Precision-Recall curve (PR-AUC) lies in its tractability for unbalanced classes. They are very similar and have been shown to contain essentially the same information, however PR curves are slightly more finicky, but a well drawn curve gives a more complete picture. The issue with PR-AUC is that its difficult to interpolate between points in the PR curve and thus numerical integration to achieve an area under the curve becomes more difficult.

Check out this discussion of the differences and similarities.

Quoting Davis' 2006 abstract:

Receiver Operator Characteristic (ROC) curves are commonly used to present results for binary decision problems in machine learning. However, when dealing with highly skewed datasets, Precision-Recall (PR) curves give a more informative picture of an algorithm’s performance. We show that a deep connection exists between ROC space and PR space, such that a curve dominates in ROC space if and only if it dominates in PR space. A corollary is the notion of an achievable PR curve, which has properties much like the convex hull in ROC space; we show an efficient algorithm for computing this curve. Finally, we also note differences in the two types of curves are significant for algorithm design. For example, in PR space it is incorrect to linearly interpolate between points. Furthermore, algorithms that optimize the area under the ROC curve are not guaranteed to optimize the area under the PR curve.

This was also discussed on Kaggle recently.

There is also some useful discussion on Cross Validated.