Comparing AUC and RMSE

Question

I want to compare the results of two papers, namely the matrix factorization techniques of Koren (2009) and Rendle et al (2009)

The first one uses RMSE values and the second one AUC. Is there any way to compare these? More specifically, I want to compare the RMSE for implicit feedback of the first paper(which is around 0.89) to the BRP-MF line on the last page of the second paper(with an AUC of 0.91-0.92).

Answer 1

You cannot compare these results at all, for at least 3 reasons:

1. different metrics
2. different datasets -- the reported results are not from experiments on the same dataset
3. different tasks -- rating prediction is different from item recommendation -- you would also not say that predicting the price of something and detecting spam are the same tasks, wouldn't you?

Why would you want to compare such different methods anyway? Maybe you want to re-read the first paper -- it uses implicit feedback as an additional signal in order to compute better rating predictions, whereas in the second paper implicit feedback is used to predict implicit feedback.