precision or recall speaks loud?

Question

Say I'm evaluating some text classification research project using two approaches 'A' and 'B'. When using approach 'A', I get a x% increase in precision while with 'B', a x% increase in recall. How can I say A or B approach better?

Answer 1

It depends on your goal. If you need the first couple of returned classes to be correct then you should go for precision, if you want to focus on returning all relevant classes then try to increase recall.

If precision and recall both matter to you then an often used measure is the F1 score which combines precision and recall into a single measure.

Answer 2

I fully agree with what @Sicco wrote.

Also, I would recommend watching this video, it's from Machine Learning course at Coursera. From the video: in some cases you can manipulate precision and recall by changing threshold. If you're not sure what's more important for you just stick to F1.