ML//evaluation//PR-AUC

2026-07-04

PR-AUC is the area under the precision-recall curve, the evaluation metric that summarizes over every threshold how pure a model's alarms are (precision) against how many real positives it catches (recall), and it is the metric that respects rare positives: failures in a fleet, defects on a line, fraud among payments.

Unlike ROC-AUC, it is sensitive to class imbalance: on a rare-event problem a high ROC-AUC can hide a useless precision. The reason sits in the denominators. ROC uses the false positive rate, false alarms divided by all negatives, and with 2,000 healthy windows per failure that denominator is so large that hundreds of false alarms barely move it; precision divides by the alarms raised, so every false alarm shows. Its baseline equals the positive rate rather than 0.5, so a PR-AUC of 0.1 can still be real signal when positives are 1 % of the data.

In practice it is computed as average precision, the precision at each step of recall weighted by how much recall that step adds, which is what scikit-learn returns under average_precision. The trapezoidal area under the curve interpolates optimistically between points and is avoided.

It compares models on the same data and nothing more. Its value moves with the positive rate, so a PR-AUC measured on a rebalanced test set, or on a plant with twice the failures of another, is not comparable across them. It also leaves open which threshold to run, which is a cost decision on the chosen model (decision threshold).

Advice26 · jul 04Read noteUse it whenever the positive class is scarce and false alarms are costly: its baseline is the positive rate, so it shows whether the alarms are worth raising exactly where ROC-AUC flatters a model.