Statistical Inference and Learning for Shapley Additive Explanations (SHAP)

arXiv:2602.10532 · stat.ML, cs.LG, math.ST, stat.TH · Submitted 2026-02-11 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Statistical Inference and Learning for Shapley Additive Explanations (SHAP)".

Jane: Here's the gist of the paper. They cover distinct, yet complementary, aspects of a single research paper concerning Statistical Inference and Learning for Shapley Additive Explanations (SHAP).

Tom: My synthesis must be precise, comprehensive, and reflect the mathematical rigor described in the technical excerpts alongside the empirical findings.

Jane: Here is the detailed summary: Comprehensive Research Summary: Statistical Inference and Learning for Shapley Additive Explanations (SHAP) This paper presents a rigorous statistical framework for both inferring quantities derived from SHAP values and learning the entire SHAP curve itself, addressing a critical gap in existing literature: the lack of statistical inference methods for derived quantities like mean absolute or mean squared SHAP values.

Tom: The core contribution lies in developing de-biased loss functions and applying sophisticated semi-parametric statistics to establish asymptotic normality and tight error bounds for learned explanations.

Title and authors: ---: Title and authors ---

Tom: So we're looking at this paper called "Statistical Inference and Learning for Shapley Additive Explanations (SHAP)" by Justin Whitehouse, Ayush Sawarni, and Vasilis Syrgkanis. It sounds super technical, but basically it’s about figuring out how to get reliable numbers from the SHAP values we already have.

Jane: Right. The title says "Statistical Inference and Learning for Shapley Additive Explanations (SHAP)". If you know the SHAP curve, that's good, but what if you want a solid number, like the average absolute value of those SHAP values? That’s where this paper steps in.

Lu: It addresses a gap where people use these local explanations but then need a global measure of feature importance to make decisions about which features matter most. This paper is building the statistical tools for that global measure.

Meng: So, it’s not just about explaining one prediction; it’s about getting a statistically sound estimate of how much each feature contributes globally. That sounds useful for deployment because you need reliable metrics to judge model performance across different scenarios, right?

Lalam: Exactly. It takes those local explanations and turns them into something you can actually trust for making big choices about a model's structure or feature selection.

---: The paper's summary ---

Tom: Okay, so the paper dives into this by suggesting a semi-parametric approach to estimate the moments of the SHAP curve, specifically the pth powers, denoted as theta p. It says they have estimators for these things and proves their asymptotic behavior based on whether p is greater than two or less than two.

Jane: That’s a big chunk of work. They handle p greater than two by using a de-biased/Neyman orthogonal score, which is pretty standard for getting good confidence intervals when you have some nuisance parameters involved.

Lu: And for when p is between one and two, where the function isn't twice differentiable, they use a smoothed surrogate function with a hyperbolic tangent to get an estimator. That handles those tricky parts of the math without getting stuck on non-differentiability issues.

Meng: So, if you’re trying to build a system that needs to know the average absolute SHAP value, this gives you the framework for doing that with statistical backing instead of just guessing based on a few samples.

Lalam: It essentially provides the math for getting confidence intervals around those global importance metrics, which is what we need when we want to make claims about feature selection.

---: The paper's improvements ---

Tom: The paper introduces some specific improvements they suggest. First, they propose using a de-biased U-statistic based on Neyman orthogonal scores for p greater than or equal to two to get asymptotically normal confidence intervals for global SHAP powers.

Jane: That’s a direct upgrade from what we have now. It moves us beyond just plug-in estimates that aren't guaranteed to be normally distributed unless you have really strong assumptions on the nuisance parameters, which they call super-parametric convergence rates.

Lu: They also suggest using that smoothed estimator for p less than two by replacing the non-differentiable functional with a smoothed alternative using a hyperbolic tangent function, which lets you do valid inference on those irregularly behaved global importance measures by tuning the smoothing parameter carefully.

Meng: From an engineering standpoint, this means we can stop relying on arbitrary smoothing choices and instead use a mathematically derived one that converges to the true power when tuned correctly.

Lalam: And they also show you how to learn the entire SHAP curve using a de-biased loss function that is Neyman orthogonal with respect to both the regression estimate and additional nuisance components, which gives high-fidelity estimates of the whole curve.

---: Conclusion ---

Tom: So, to wrap up, these improvements give us statistically sound ways to estimate the moments of the SHAP curve and learn the curve itself using a de-biased loss function. This means we can finally move past just having an unbacked plug-in estimate for global importance.

Jane: The paper gives us tight error bounds, like - phi a squared r squared n + - g zero four G, which tells us exactly how the estimation error scales with the sample size and the complexity of those nuisance parameters.

Lu: The theoretical guarantees are pretty solid, especially with that concentration inequality bounding A by r n squared + r two/n, confirming that the learned curve error is controlled by both sample size terms and nuisance parameter errors <ref:2602.10532#pg2>.

Meng: For deployment, this means we can use these learned curves to perform robust feature selection or understand global contributions because we know how accurate those estimates are relative to the true SHAP curve.

Lalam: Ultimately, this paper gives us the tools to be much more certain about what features truly drive model behavior when we look at global importance metrics derived from SHAP.

Tom: So that’s it for this session on "Statistical Inference and Learning for Shapley Additive Explanations (SHAP)". We’ll catch some of those other papers next time.

The paper's summary: Tom: So we’re looking at this paper, "Statistical Inference and Learning for Shapley Additive Explanations (SHAP)." It’s basically about giving us solid math to back up the SHAP values we get from AI models.

Jane: Right. The main idea is that if you have those local explanations, like SHAP values, you need a better way to estimate global things—like the average importance of a feature—and this paper builds the statistical framework for that.

Lu: They tackle two big problems there. First, they provide estimators for the moments of those SHAP curves, which are called theta-p. They use different math depending on whether p is bigger than two or smaller than two.

Tom: Okay, so when p is bigger than two, they use a de-biased score and prove it’s asymptotically linear. That means if you get a big enough sample size, the estimator gets really close to the true value without needing super complex assumptions about how fast those other model parts are converging.

Jane: And for p less than two, which is where things get tricky because the function isn't always smooth, they use a smoothed version with a hyperbolic tangent to handle that irregularity. That lets us get some kind of valid inference on those less-than-two importance measures without getting stuck on non-differentiability.

Lu: The real meat is in the learning part. They introduce this new way to do loss functions, called Neyman orthogonal loss functions. This orthogonality helps them minimize the loss and actually gets you a de-biased estimate for the whole SHAP curve, not just some rough guess.

Meng: So it’s not just about getting one number right; it’s about learning the entire map—the whole SHAP curve—using a loss function that is mathematically structured to be accurate. That sounds like exactly what we need when we're trying to deploy these explanations reliably in production where accuracy matters at scale.

Tom: Exactly. They give us rigorous error bounds too, showing how the estimation error scales with the sample size and how much those other model parts are drifting around. It’s not just "it works"; it’s "it works this many times."

Jane: So what does this change for someone who just uses these explanations to make decisions? It means we can move past those simple plug-in estimates that don't have good confidence intervals, giving us the tools to actually claim statistical significance about which features are globally important.

Lu: And the experiments they did showed that these de-biased methods are way better than naive learners, especially when you look at Mean Squared Error across different training setups. The empirical results back up the theoretical claims pretty well.

Tom: So we’ve got a solid statistical foundation, a learning framework that's designed to be unbiased, and empirical proof it performs better than the old ways. It gives us a way to trust our global feature importance metrics derived from SHAP values.

Jane: It moves global feature importance from being just an educated guess to something you can actually quantify with confidence intervals based on the data you have. This sets a new bar for how we interpret these explanations across different applications.

The paper's improvements: Tom: So we’re looking at what they suggest doing next, what improvements they are pushing for in this SHAP paper. It seems like they’re not just settling for one way to do things, but trying to build a whole toolkit of better tools.

Jane: Right. They are suggesting using that de-biased U-statistic approach we talked about earlier for p greater than two, which would give us those asymptotically normal confidence intervals for global SHAP powers.

Lu: That’s the idea—using the Neyman orthogonal scores to get those normal distributions, which is huge because it lets practitioners actually test if a feature's importance is statistically significant or just noise.

Tom: So that means we can finally stop guessing about feature selection metrics and start making real statistical claims about what drives model behavior. That’s a big mechanical win for us when we’re trying to justify model changes to stakeholders.

Meng: And they also propose using that smoothed estimator for p less than two, specifically with a hyperbolic tangent function instead of just some arbitrary smoothing parameter. That should give us valid inference on those tricky, non-differentiable global importance measures.

Jane: It means we don't have to pick an arbitrary smoothing value; we can tune it carefully so the estimate converges to the true, unsmoothed power. That takes a lot of guesswork out of the process.

Lu: And they also suggest using that de-biased loss function for learning the curve itself, making it orthogonal not just to one nuisance parameter, but to both the regression estimate and other factors. This aims for higher fidelity on the entire SHAP curve estimation.

Tom: So we’re talking about a whole system now: better estimators for individual powers, a way to learn the whole curve with an orthogonal loss function, and formal tests to check if our results are statistically sound. It’s moving from just "this is good" to "this is statistically validated."

Jane: It gives us the tools to build systems where we can robustly understand why a model makes certain decisions by getting reliable, statistically backed global feature importance metrics. This moves the whole field toward more rigorous interpretation of AI outputs.

Conclusion: Tom: So to wrap up, we’ve covered how this paper on "Statistical Inference and Learning for Shapley Additive Explanations (SHAP)" gives us the statistical rigor to estimate those moments of SHAP curves and learn the curve itself with de-biased loss functions.

Jane: It really gives us confidence that when we use AI explanations, especially for global feature importance, we can get statistically sound numbers instead of just relying on rough plug-in estimates.

Lu: The error bounds they derive are pretty tight, showing exactly how the estimation error scales with the sample size and the complexity of those other model parameters. It’s a solid theoretical guarantee on accuracy.

Meng: For me, it means we can stop worrying about whether our explanation metrics are just random noise from a small training set; if we use this de-biased approach, we have a much better chance of deploying explanations that actually reflect the model's true behavior at scale.

Lalam: I’m pretty sure this direction is the right one because it connects the deep mathematical work directly to making our AI outputs more trustworthy for real-world decisions.

Tom: Exactly. It’s about moving from just having an unbacked plug-in estimate for global importance to something that has a formal statistical guarantee behind it, which is what we need when we're trying to justify model changes or feature selection.

Jane: So the big picture here is that this work provides the necessary mathematical machinery to make AI explanations more reliable and interpretable across all those global metrics we care about.

Lu: The limitation they mention, though, is that it still depends on assumptions about how fast those nuisance parameters are converging, so you still need to be careful about those convergence rates.

Tom: Right. It’s not a magic bullet; it’s a framework that lets us build more robust systems on top of existing model training techniques.

Jane: So next time, we'll be looking at papers that tackle uncertainty in medical image segmentation, which is a totally different kind of statistical challenge than the one we just discussed.

Justin Whitehouse, Ayush Sawarni, Vasilis Syrgkanis

Stanford University

stat.ML, cs.LG, math.ST, stat.TH

Submitted: 2026-02-11

Updated: 2026-10-17

Comments: 48 pages, 1 figure

Code: https://github.com/shap/shap

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 91/100

The gist: They cover distinct, yet complementary, aspects of a single research paper concerning Statistical Inference and Learning for Shapley Additive Explanations (SHAP).

Key concepts

De-biased Loss Functions: These are specialized loss functions designed to minimize errors in estimating the true SHAP curve rather than just fitting the data. They use Neyman orthogonality, ensuring that minimizing this specific loss leads directly to an unbiased estimate of the true Shapley values.
Asymptotic Normality: This mathematical property proves that as you collect more data, your statistical estimator for SHAP quantities (like mean absolute SHAP) will follow a normal distribution. This allows researchers to construct reliable confidence intervals and determine the statistical significance of feature importance metrics.
Neyman Orthogonality: This concept ensures that the chosen loss function is statistically independent of the nuisance parameters being estimated alongside the main model. By making them orthogonal, minimizing this loss isolates the error related only to estimating SHAP, leading to a more accurate and unbiased result.

Terminology

Explainability — 1× in this paper · explained in 4 episodes
Feature Importance — 1× in this paper · explained in 2 episodes
Feature Selection — 1× in this paper · explained in 4 episodes
Model-Agnostic — 1× in this paper · explained in 2 episodes
Shapley values — 1× in this paper · explained in 2 episodes

Summary

They cover distinct, yet complementary, aspects of a single research paper concerning Statistical Inference and Learning for Shapley Additive Explanations (SHAP). My synthesis must be precise, comprehensive, and reflect the mathematical rigor described in the technical excerpts alongside the empirical findings.

Here is the detailed summary:

Comprehensive Research Summary: Statistical Inference and Learning for Shapley Additive Explanations (SHAP)

This paper presents a rigorous statistical framework for both inferring quantities derived from SHAP values and learning the entire SHAP curve itself, addressing a critical gap in existing literature: the lack of statistical inference methods for derived quantities like mean absolute or mean squared SHAP values. The core contribution lies in developing de-biased loss functions and applying sophisticated semi-parametric statistics to establish asymptotic normality and tight error bounds for learned explanations.

I. Theoretical Contributions: Statistical Inference and Asymptotic Validity

The theoretical foundation of the paper is built upon constructing asymptotically valid estimators for various moments of the SHAP curve phi a(X). The methodology employs a semi-parametric approach, treating the SHAP curve as a nuisance function that must be estimated from data, which allows for reliable confidence interval construction.

A. Estimating Powers of SHAP Values (theta p):

The authors provide estimators for the p-th powers of the SHAP curve, denoted as theta p:= E[phi a(X) p], and establish their asymptotic behavior based on the value of p:

For p at least 2: An estimator (theta b n) is constructed using a de-biased/Neyman orthogonal score, relying on two independent samples (Z and Z'). Theorem 3.3 proves that under specific conditions on the convergence rates of nuisance parameter estimates (phi bn, mu bn, alpha bn, and gamma bn), this estimator is asymptotically linear about the true value theta p.
For 1 at most p < 2: Since times p is not generally twice differentiable in this range, the authors address this irregularity by introducing a smoothed surrogate function, theta p, beta:= E[phi p, beta(phi a(X))]. They then construct a de-biased score and utilize U-statistics applied to this smoothed surrogate. Theorem 3.7 establishes asymptotic normality for the estimator theta b n, showing that the normalized error converges to a term involving the empirical process: sqrt n (theta b n - theta p) = sqrt 1/n P p(Z i) + o(P(1)).

B. Statistical Learning of the SHAP Curve (phi a(X)):

The paper introduces a statistical learning approach for estimating the entire SHAP curve phi a(X) using an orthogonal statistical learning framework, drawing inspiration from Foster and Syrgkanis [20].

De-biased Loss Functions: The key innovation here is the definition of Neyman orthogonal loss functions. Proposition 4.2 defines partial loss functions S and an aggregated pointwise loss L(phi; mu, zeta) that are Neyman orthogonal in the sense of Definition 4.1. This orthogonality ensures that minimizing this loss yields a de-biased estimator for the true SHAP curve phi a(X).
Error Bounds: The analysis establishes rigorous error bounds on the learned curve relative to the oracle curve phi a. Theorem 4.4 demonstrates that under specific assumptions (B1-B5), the squared L squared distance between the estimated and true SHAP curves is bounded:

| - phi a| 2 L squared = O(r 2n) + O| - g 0| 4 G

This result confirms that the estimation error scales appropriately with sample size (n) and the complexity of the nuisance parameters (G).

II. Mathematical Analysis: Concentration Inequalities and Error Propagation

The theoretical guarantees rely heavily on advanced concentration inequalities to bound empirical process differences.

Loss Decomposition: The total loss function is decomposed into a squared term (sq) and linear correction terms (lin), allowing for a structured analysis of the error propagation: = sq + lin.
Empirical Process Bounding: The analysis proceeds by bounding the empirical process difference, leading to a concentration inequality (Equation 17). By substituting this bound back into the loss expression, the authors derive a final concentration inequality for the error term A:

A = P (Z, Z';) r n| | squared + r 2/n

Final Error Rate: By applying an inequality relating r n| | squared to the true error phi and the nuisance parameter error | - g 0| 4 G, the paper concludes with a tight, high-probability bound on the squared difference between learned and true SHAP values:

| - phi a| squared r squared n + | - g 0| 4 G

This confirms the claimed O(times) rate, demonstrating that the estimation error is controlled by a term proportional to sample size (r squared n) plus a term dependent on the learned nuisance parameters (| - g 0| 4 G).

III. Experimental Validation and Empirical Findings

The theoretical framework is validated through empirical studies comparing different learning strategies for the SHAP curve:

Data Generation: The study utilizes a synthetic regression problem with d=5 covariates, generating data from a specific positive-definite covariance matrix and an additive-noise model.
Learner Comparison: The experiments compare a naive learner against an orthogonal (debiased) learner. Furthermore, the performance is assessed across two training regimes: split and reuse.
Key Empirical Results: The results consistently demonstrate that the orthogonalization technique significantly improves accuracy. Specifically, for instance, the mean Mean Squared Error (MSE) for the orthogonal+reuse method showed a substantial decrease from 0.681 at n=500 to 0.141 at n=2000. In stark contrast, the naive+split baseline exhibited a much poorer performance, decreasing from 1.745 to an unstated higher value over the same range. This empirical evidence supports the theoretical claim that de-biasing and orthogonalization are crucial for achieving accurate SHAP curve estimation in practice.

**In summary, this paper provides both a robust theoretical justification—grounded in semi-parametric statistics, U-statistics, and Neyman orthogonality—for constructing statistically valid estimators of SHAP moments and for developing a statistically sound learning framework to estimate the entire SHAP curve. The theoretical guarantees are strongly supported by empirical results showing superior performance for de-biased methods.

Improvements for AI systems

Asymptotically normal confidence intervals for global SHAP powers can be constructed using a de-biased U-statistic based on Neyman orthogonal scores when p greater than or equal to 2. This allows practitioners to reliably estimate statistical significance for feature selection metrics like mean absolute SHAP, moving beyond naive plug-in estimates that are not asymptotically normal unless super-parametric convergence rates hold.
Smoothed estimators for the pth power of the SHAP curve can be used when p is less than 2 by replacing the non-differentiable functional with a smoothed alternative using a hyperbolic tangent function. This enables valid inference on irregularly behaved global importance measures by carefully tuning the smoothing parameter to ensure convergence to the true, unsmoothed power.
Model-agnostic variable importance metrics can be learned via empirical risk minimization using a de-biased loss function that is Neyman orthogonal with respect to both the regression estimate and additional nuisance components. This provides high-fidelity estimates of the entire SHAP curve, allowing for robust feature selection or understanding of global feature contributions based on model fitting.
A central limit theorem can be established for smoothed U-statistics applied to SHAP scores, showing that the scaled difference between a plug-in estimate and the true population power converges to a normal distribution under appropriate conditions on nuisance estimation rates. This provides a formal statistical tool for quantifying the uncertainty in global feature importance estimates derived from learning algorithms.
A generic bound on the L2 error of an estimated SHAP curve can be derived based on empirical risk minimization, showing that the error is bounded by terms related to the complexity of the function class and the estimation error of nuisance parameters. This provides a theoretical guarantee on how accurately a learned SHAP curve approximates its true underlying value.