A probability calibrator for machine-learning classifiers

arXiv:2302.10578 · cs.LG, physics.data-an, stat.ML · Submitted 2023-02-21 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "A probability calibrator for machine-learning classifiers".

Jane: Here's the gist of the paper. The gist: The main idea of the present work is to calculate probabilities conditional not on the features, but on the trained classifier’s output, providing an output-to-probability ‘transducer’ that can be applied to all future outputs of the classifier.

Tom: The inadequacy of common classification approaches The paper argues that in fields like medicine or drug discovery, a classification's ultimate goal is not to guess a class, but to choose the optimal course of action among a set of possible ones, which requires sensible probabilities conditional on the features being computationally almost impossible to find <ref:2302.10578#pg2>.

Jane: The machine-learning mindset often reflects simpler needs and contexts, whereas medical or drug classification problems differ vastly in their gains and losses <ref:2302.10578#pg3>.

Tom: Specifically, the utility of correct and incorrect classifications is vastly different between a 'cat vs dog' image classification problem and a 'disease I vs disease II' medical classification <ref:2302.10578#pg4>.

Paper summary: Jane: So wrapping this up for "A probability calibrator for machine-learning classifiers," the authors are showing that by calculating the probability conditional on the classifier's output, rather than just its input features, you get a mechanism to make choices that align better with real-world goals

reference:two thousand three hundred two point one zero five seven eight#pg1: .

Lu: It’s essentially a one-time calculation that creates an output-to-probability transducer which can be applied to all future outputs of the classifier, and combining this with expected utility maximization always leads to better performance across all problem-dependent utilities

reference:two thousand three hundred two point one zero five seven eight#pg47: .

Meng: The implication for us in production is that we gain an evaluation of the whole classifier, including a measure of uncertainty about that evaluation, which helps us choose the right algorithm for our specific task

reference:two thousand three hundred two point one zero five seven eight#pg7: .

Tom: And they also give us this generative mode where we can use those probabilities with real population prevalences to get a better estimate of the decision probability in skewed training sets

reference:two thousand three hundred two point one zero five seven eight#pg32: .

Jane: It’s about moving beyond just guessing the most likely thing toward making the best possible action, even when dealing with complex, high-stakes problems where accuracy alone doesn't define success

reference:two thousand three hundred two point one zero five seven eight#pg4: .

Conclusion: Tom: So, we're wrapping up this look at "A probability calibrator for machine-learning classifiers." Essentially, the paper is about fixing something fundamental in how we use these models to make decisions in real life.

Jane: Right. It talks about this idea of not just guessing a class from the input features, but figuring out what the actual probability of that class *is* based on what the classifier has already decided.

Lu: The core mechanism is this output-to-probability transducer, which you can apply to any classifier you've trained. It’s a one-time calculation done once, and then it lets you map whatever the model spits out—that raw output—onto a calibrated probability.

Meng: From an engineering standpoint, that sounds like something that could save us from deploying models with totally unreliable confidence scores across the board. We deal with uncertainty in production constantly.

Tom: Exactly. Because they couple that transducer with expected utility maximization, it lets you actually choose the best action among all your options using those calibrated probabilities instead of just picking the highest output score.

Jane: So for someone listening, this means we're not just looking at a number like ninety-five percent confidence; we’re getting a better idea of what that ninety-five percent actually means in terms of making the right choice.

Lu: And it does this across all problem-dependent utilities. They show that this augmentation always improves results, even when you have really weird datasets, like highly imbalanced ones.

Meng: That’s important for us because we often deal with biased training data where the standard metrics just look good on paper but fail when you push them to real traffic.

Tom: So it seems the authors aren't claiming a magical fix, but rather a consistent way to improve performance across all those different ways you might want to evaluate success.

Jane: It’s about moving from just accuracy numbers to actual decision quality. And that leads us next into how this technique handles situations where the training data doesn't match the real world, which is something I think we should look at next.

K. Dyrland, A. S. Lundervold, P.G.L. Porta Mana

Western Norway University of Applied Sciences · Haukeland University Hospital

cs.LG, physics.data-an, stat.ML

Submitted: 2023-02-21

Updated: 2026-10-18

Comments: 54 pages, 10 figures. V2: rewritten title, added package references

Project page: https://probml.github.io/pml-book/book0.html

License: http://creativecommons.org/licenses/by/4.0/

The gist: The gist: The main idea of the present work is to calculate probabilities conditional not on the features, but on the trained classifier’s output, providing an output-to-probability

Key concepts

Probability Transducer: This is a mechanism that maps a classifier's raw output (like a score) directly to the probability of each class. It does this using exchangeable 'transducer-calibration data.' This calculation is done only once, making it computationally cheap and applicable to any existing classifier without retraining.
Augmentation: This is the core method where the transducer and expected utility maximization are combined. Instead of just picking the most likely class, augmentation uses a utility matrix to find the decision that maximizes expected utility, leading to better results than standard methods.
Expected Utility Maximization: This principle guides decision-making by calculating the expected payoff for every possible action. By multiplying a utility matrix (which defines how much you gain or lose from each outcome) by the calculated class probabilities, it selects the decision with the highest overall expected value.
Generative Mode: This capability allows the system to calculate not just how likely a class is given an output, but also the inverse: how likely that output would have been if you knew which class was true. This is useful when training data frequencies don't match real-world population frequencies.

Summary

The gist: The main idea of the present work is to calculate probabilities conditional not on the features, but on the trained classifier’s output, providing an output-to-probability ‘transducer’ that can be applied to all future outputs of the classifier.

The inadequacy of common classification approaches

The paper argues that in fields like medicine or drug discovery, a classification's ultimate goal is not to guess a class, but to choose the optimal course of action among a set of possible ones, which requires sensible probabilities conditional on the features being computationally almost impossible to find <ref:2302.10578#pg2>. The machine-learning mindset often reflects simpler needs and contexts, whereas medical or drug classification problems differ vastly in their gains and losses <ref:2302.10578#pg3>. Specifically, the utility of correct and incorrect classifications is vastly different between a 'cat vs dog' image classification problem and a 'disease I vs disease II' medical classification <ref:2302.10578#pg4>. The target of a classification is not what’s most probable, but what’s optimal <ref:2302.10578#pg4>.

The probability transducer

The core idea is to calculate the probability P(class of new unit output for new unit) (2), which acts as an output-to-probability transducer that maps the algorithm’s raw output to a probability <ref:2302.10578#pg8>. This transducer has a low computational cost, can be applied to all commonly used classifiers and simple regression algorithms, and does not need any changes in algorithm architecture or in training procedures <ref:2302.10578#pg6>. It gives a quantification of how much the probability would change if there were further data to calculate the transducer <ref:2302.10578#pg7>.

Calibration data and calculation

To calculate these conditional probabilities (2), it is necessary to have examples of further pairs (class of unit, output for unit) that are exchangeable with the new unit, which are termed ‘transducer-calibration data’ <ref:2302.10578#pg9>. The probability we want to calculate is therefore P(class of new unit output for new unit, classes & outputs of calibr. data) (3). This joint probability is calculated using standard non-parametric Bayesian methods <ref:2302.10578#pg12>. The resulting parameters are determined by the test data D and can be used to compute any of the probabilities (5), (6), or (7) as needed, with computations taking less than a second <ref:2302.10578#pg13>.

Augmentation of a classifier

The combination of the probability transducer and the rule of expected-utility maximization is called the ‘augmentation’ of a classifier <ref:2302.10578#pg4>. The procedure involves:

Feeding the unit’s features to the classifier, which outputs the real value y <ref:2302.10578#pg15>.
Calculating p(c y), for each value of c, from formula (7a), using the parameters stored in memory <ref:2302.10578#pg15>.
Determining the expected utility U¯ i of decision i by the matrix product of an appropriate utility matrix (Ui c) and the column vector p c <ref:2302.10578#pg9>.
Choosing the decision i∗ having largest U¯ i, according to the principle of maximum expected utility <ref:2302.10578#pg10>.

Benefits and uses of the transducer

The probability transducer provides several automatic benefits:

(i) a quantification of how much the probability would change if we had further data to calculate the transducer <ref:2302.10578#pg7>.

(ii) an evaluation of the whole classifier – including an uncertainty about such evaluation – that allows us to compare it with other classifiers and choose the optimal one <ref:2302.10578#pg7>.

(iii) the possibility of using the algorithm in a ‘generative mode’, useful if the training dataset is biased <ref:2302.10578#pg7>.

Assessment of performance

The augmentation improves the performance of each algorithm in all cases, and this improvement also occurs in case IV for the random forest, where the standard method already had an extremely high performance <ref:2302.10578#pg24>. The augmentation always leads to an improved utility yield, especially for those cases where the standard method has a low or high performance <ref:2302.10578#pg45>. The calculation of the expected utility automatically produces a probability distribution of the possible long-run yields the algorithm could give <ref:2302.10578#pg31>.

Generative mode and population mismatch

The transducer parameters allow us to calculate not only the ‘discriminative’ probability p(c y) but also the inverse, ‘generative’ probability p(y c) <ref:2302.10578#pg34>. This generative mode is useful when the training and calibration sets do not have the same class frequencies as the real population on which the classifier will be employed <ref:2302.10578#pg31>. By using these generative probabilities with population prevalences, we can calculate p(c y, prevalences) <ref:2302.10578#pg32>.

Future directions

Future work plans include constructing a probability transducer that takes the output from several classifiers at once to optimize ‘ensembling’ and assessing the best way to split available data into training and calibration sets for optimal mutual information <ref:2302.10578#pg46>. The paper concludes by showing that the augmentation leads to improved utility yields across all possible utility matrices, especially when compared to standard methods <ref:2302.10578#pg47>.

Improvements for AI systems

Augmentation of Classifiers: The system can choose the optimal course of action among a set of possible ones by means of expected-utility maximization, as described by the principle bearing this name. This allows for decisions that are not in one-one correspondence with classes, such as discarding an ineffective drug candidate based on economic losses.
Uncertainty Quantification: The system can provide a quantification of how much the probability would change if we had further data to calculate the transducer, which helps in evaluati[ng] the whole classifier – including its uncertainty about such evaluation. This output is used for algorithm selection and provides a measure of confidence in the model's performance.
Generative Mode Application: The system can be used in a generative mode where it calculates the probability of features conditional on class, which is useful if the training dataset is biased. This enables the calculation of probabilities even when class frequencies in real-world data differ from calibration data, using Bayes’s theorem to find p(c y, prevalences).
Dynamic Utility Adaptation: The system can handle situations where utilities change on the fly because the utilities Uic used in step 3 can either be the same for each new unit, or different from unit to unit, without requiring changes to the algorithm's architecture or training procedures.
Optimal Threshold Selection: The augmentation procedure automatically optimizes a decision threshold based on problem-dependent utilities, which can lead to better results than standard methods, as seen in Case IV where it leads to the confusion matrix [where] all items were classified as '0', 'inactive'.

Abstract

In fields such as medicine and drug discovery, the ultimate goal of a classification is not to guess a class, but to choose the optimal course of action among a set of possible ones, usually not in one-one correspondence with the set of classes. This decision-theoretic problem requires proper probabilities for the classes. Proper probabilities conditional on the features are computationally almost impossible to find in many important cases. The main idea of the present work is to calculate probabilities conditional not on the features, but on the trained classifier's output. This calculation is cheap, needs to be made only once, and provides an output-to-probability "calibrator" or "transducer" that can be applied to all future outputs of the classifier. In conjunction with problem-dependent utilities, the probabilities of the transducer allow us to find the optimal choice among the classes or among a set of more general decisions, by means of expected-utility maximization. This idea is demonstrated in a simplified drug-discovery problem with a highly imbalanced dataset. The transducer and utility maximization together always lead to improved results, sometimes close to theoretical maximum, for all sets of problem-dependent utilities. The one-time-only calculation of the transducer also provides, automatically: (i) a quantification of the uncertainty about the transducer itself; (ii) the expected utility of the augmented algorithm (including its uncertainty), which can be used for algorithm selection; (iii) the possibility of using the algorithm in a "generative mode", useful if the training dataset is biased.