ML//evaluation//class imbalance

Class imbalance is the situation in which one class massively outnumbers the other, so that a model can score well by mostly ignoring the rare class. In industrial failure prediction it is the normal case: with one failure per 2,000 time windows, a model that always answers *normal* is 99.95 % accurate and catches nothing (base rate).

Class imbalance is the situation in which one class massively outnumbers the other, so that a model can score well by mostly ignoring the rare class. In industrial failure prediction it is the normal case: with one failure per 2,000 time windows, a model that always answers normal is 99.95 % accurate and catches nothing (base rate).

It breaks accuracy and ROC-AUC as measures, since both are dominated by the abundant negatives; precision and recall, PR-AUC and the behavior against the base rate are the ones to read instead.

Even a good model concentrates the rare class without purifying it: most alarms stay false because most cases are non-events. That can be perfectly profitable. With an inspection costing 2,000 euros and a compressor failure 200,000, the break-even probability is about 1 %, so a detector whose alarms are real 9 % of the time is an excellent reason to inspect (decision threshold).

It is handled at training time with class-weighted cross-entropy, focal loss, or resampling. All three push the predicted probabilities toward the rare class, so a model trained this way needs probability calibration before its output is compared with a cost threshold.

It starves validation more than training. With ten failures in the data, each fold of cross-validation sees two or three and the score jumps with every one of them; the folds are stratified, and the number of independent failure events, never the number of rows, says how far the score can be trusted.