will increasing threshold always increase precision?

Question

here precision at threshold 0.85 > precision at threshold 0.90. shouldnt it be the other way round? increasing threshold will reduce False positive and precision will be greater than before?

Answer 1

here precision at threshold 0.85 > precision at threshold 0.90. shouldnt it be the other way round? increasing threshold will reduce False positive and precision will be greater than before?

Precision is $\frac{\text{TP}}{\text{TP}+\text{FP}}$

Both $\text{TP}$ and $\text{FP}$ are reduced when you increase the threshold. If both decrease in proportion to the current precision (i.e. they are spread evenly at each confidence value), then precision will remain the same. Most models on most datasets will tend to increase precision as the threshold increases, at least initially (e.g. moving from 0.5 to 0.6) as false positives may commonly be found as uncertain edge cases with low confidence, i.e. false positives tend to occur more frequently at low confidence, so increasing threshold will exclude a higher ratio of false positives than true positives than the current precision. However, there is no guarantee of that.

The value of precision will vary in practice depending on what the model predicted for each example. If you have a cluster of highly confident false positives, they can cause precision to drop as threshold grows, until they get excluded. The most extreme example would be where the most confident classification is incorrect, in which case the highest possible threshold will score zero precision.