Neighborhood Smoothing for Calibration

arXiv:2610.09020 · cs.LG · Submitted 2026-10-06 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Neighborhood Smoothing for Calibration".

Jane: The gist: Graph smoothing, introduced as NeSCal, is a train-time regularizer that penalizes Jensen–Shannon divergence between neighboring predictive distributions to encourage similar predictions across representation space,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we’re wrapping up the overview of "Neighborhood Smoothing for Calibration". We talked about how they propose graph smoothing as a general principle for calibration and introduced NeSCal, which penalizes the Jensen–Shannon divergence between predictive distributions of neighboring samples.

Jane: That was just setting the stage, Tom. To recap, they are trying to address the problem that neural networks are often miscalibrated with a tendency to overconfidence after training >

Lu: The core thesis is that existing methods mostly modify task losses or calibration penalties, leaving neighborhood structure in learned representations completely underexploited >

Tom: They argue that we should encourage similar predictive distributions across neighboring samples in the representation space itself, which they formalize as graph smoothing >

Jane: So, this isn't just about tweaking a loss function; it's about regularizing the model to enforce local consistency based on how those predictions relate to each other in the learned space >

Tom: They analyzed the effects of this smoothing by deriving bounds that link predictive divergence between neighboring samples to local confidence variation and error propagation along paths in that graph >

Jane: So, they are proving a mathematical connection: if the predictions are smooth between neighbors, then we get a bound on how much calibration error can travel through those connections >

Lu: They also characterize precisely when this smoothing improves calibration and when it fails to do so by looking at the true correctness probabilities of the samples >

Jane: Which is really important because it tells us that not every local smoothness translates into global calibration improvement, which is a very realistic constraint for any real system >

Tom: And they proposed NeSCal as the specific tool to penalize this Jensen–Shannon divergence between neighboring predictive distributions, added to the task loss via a weighting term at train time >

Lu: So it’s an idea that we can use to structure our training process around neighborhood consistency in the representation space rather than just relying on external adjustments >

Jane: It seems like they are moving toward a more principled way of doing train-time calibration by integrating structural information directly into the optimization objective >

Tom: And this approach has been tested empirically, showing that it improves predictive quality across standard calibration benchmarks and is complementary to post-hoc methods >

Jane: So, in short, the paper "Neighborhood Smoothing for Calibration" shows that incorporating graph structure directly into the training process offers a principled way to get better calibration results without relying solely on external fixes >

Conclusion: Tom: So we’ve gone through the overview of "Neighborhood Smoothing for Calibration". The paper is by Idan Horowitz and Avigdor Gal from the Faculty of Data and Decision Sciences Technion in Haifa >

Jane: That was a lot to process. To put it simply, this research suggests that training our AI models should focus on learning representations where neighboring samples have similar predictive behaviors >

Lu: It’s about structuring the training process so that we are regularizing for neighborhood consistency in the representation space instead of just patching errors later >

Tom: The authors proposed NeSCal as a graph-based regularizer that penalizing Jensen–Shannon divergence between those neighbors, and they showed this helps improve predictive quality on benchmarks >

Jane: So the implication for anyone listening is that if you’re building models, thinking about how you structure your training around these neighborhood relationships could be a way to get better calibration results without needing complicated post-hoc tuning >

Lu: It moves us toward using structural information as a primary driver for achieving better calibration, which is what they are aiming for with this work >

Tom: And it’s important to remember that while the method has its limitations, like it doesn't always improve calibration depending on the specific data structure >

Jane: That’s right. The findings support using neighborhood smoothing as a framework for train-time calibration and predictive improvement, but we need to be careful where we apply it >

Idan Horowitz, Avigdor Gal

Technion

cs.LG

Submitted: 2026-10-06

Updated: 2026-10-06

Code: https://github.com/idanh8/nescal_paper_code

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 91/100

The gist: The gist: Graph smoothing, introduced as NeSCal, is a train-time regularizer that penalizes Jensen–Shannon divergence between neighboring predictive distributions to encourage similar predictions

Key concepts

Graph Smoothing: This is a general principle for training where similar predictions across neighboring data points are encouraged. It treats the samples as nodes in a graph, and the connections (edges) represent how similar their learned representations are. The goal is to make predictions consistent between these neighbors.
ΝmodelNoSpace: This is the specific graph-based regularizer proposed by the authors. It adds a penalty term to the training loss that measures the Jensen--Shannon divergence between the predictive distributions of neighboring samples in representation space. This forces similar representations to yield similar predictions.
Jensen--Shannon Divergence (JSD): JSD is a mathematical measure used here to quantify how different two probability distributions are. In this context, it measures the difference between the predicted probabilities of two neighboring samples. Penalizing high JSD encourages the model to produce more similar predictions for samples that are close in the representation space.

Terminology

Calibration — 1× in this paper · explained in 8 episodes

Summary

The gist: Graph smoothing, introduced as NeSCal, is a train-time regularizer that penalizes Jensen–Shannon divergence between neighboring predictive distributions to encourage similar predictions across representation space, and empirical analysis shows it improves predictive quality in many settings.

How it works

Graph smoothing is introduced as a general principle for train-time calibration, which encourages similar predictive distributions across neighboring samples in representation space (Page 1). This is instantiated as NeSCal, a graph-based train-time regularizer that penalizes the Jensen–Shannon divergence between neighboring predictive distributions, added to a task loss through a single weighting term at no inference-time cost (Page 2). The objective is formulated as min fθ L(θ) = Ltask(θ) + λ Rn(p), where Rn(p) = P i,j Wij D(pi∥pj) is the neighborhood consistency regularizer (Page 3).

The analysis establishes bounds connecting predictive divergence to local confidence variation and the propagation of pointwise calibration error along paths in the graph (Page 3). Proposition 1 states that under the standing domination condition, Ecalj ≤ Ecali + 1/2√CD q D(pi∥pj) + ai − aj, which bounds the propagation of pointwise calibration error (Page 3). Theorem 1 extends this to paths through the graph, showing that Ecalv ≤ Ecals + vuutr/4CD Xrl=1 Dl + av − as, where Dl is the divergence along the path (Page 3).

Theoretical Guarantees

The paper derives several key theoretical relationships to characterize when smoothing improves calibration (Page 3). Proposition 4 expresses Rn(p) as a weighted graph-Laplacian quadratic form, giving it a geometric interpretation as a smoothness penalty over some representation manifold M ⊆ Z (Page 15). Lemma 1 establishes that if D dominates squared total variation with constant CD, then pi − pj1 ≤ 1/√CD q D(pi∥pj) (Page 17). Proposition 5 shows that confidence smoothness holds, bounding the difference in confidences: ci − cj ≤ 1/2√CD q D(pi∥pj) (Page 18).

When Smoothing Improves Calibration

The paper identifies conditions under which smoothing can or cannot improve calibration (Page 3). Proposition 2, the Smoothing Obstruction, shows that for every edge (i, j) ∈ E, Ecali + Ecalj ≥ ai − aj − 1/2√CD q D(pi∥pj), meaning smoothing is obstructed from calibrating both endpoints unless ai = aj (Page 3). Proposition 3 characterizes when averaging reduces expected squared error, stating that it occurs iff (ai − aj)2 < E[(ei − ej)2, where ei and ej are the pointwise errors, indicating that smoothing helps only when neighboring errors differ more than neighboring correctness probabilities do (Page 18).

Empirical Results

Empirical analysis shows that NeSCal improves predictive quality across standard calibration benchmarks (Page 5). NeSCal improves NLL over CE in all eleven main benchmark settings and Brier score in ten of eleven settings, and ranks first or second among the evaluated methods in seven of eight image and tabular settings (Page 6). After temperature scaling, NeSCal attains the lowest NLL in seven of those eight settings (Page 7).

Robustness and Limitations

The study investigates robustness under input corruption and label noise (Page 34). NeSCal improves corrupted NLL and ECE over CE in four of five image settings, with higher accuracy in all five, while on Tiny-ImageNet it attains the highest reported SVHN AUROC (Page 35). However, gains depend on evaluation conditions and metric; for instance, improvements in NLL do not always accompany improved accuracy under label noise (Page 34). The gains are most useful when disagreement is both removable and supported by an informative, learnable representation (Page 8).

Conclusion

The findings support neighborhood smoothing as a framework for train-time calibration and predictive improvement (Page 8). NeSCal is expected to improve calibration to the extent that its graph connects points which genuinely share a true probability of correctness, a condition the design choices of Section 4.2 are intended to promote (Page 19). Future work should consider further development of the graph smoothing framework and focus on encouraging smoothing in regimes where it helps (Page 8). The paper concludes that representation-based neighborhoods yield lower NLL than random neighborhoods, and gains persist under a common temperature-scaling procedure (Page 8).

Improvements for AI systems

The NeSCal regularizer penalizes Jensen–Shannon divergence between neighboring predictive distributions to encourage similar predictive distributions across neighbors in representation space, which is a general principle for train-time calibration. This allows the system to constrain local confidence variation by enforcing that nearby samples in representation space should, when they correspond to similar underlying inputs, receive similar predictions.
The system can be improved by dynamically adjusting the regularization strength using a schedule informed by validation NLL, as Activation epoch and λ are selected jointly by validation NLL; the theory motivates the schedule without determining it. This dynamic approach ensures that smoothing is applied when it is most beneficial, aligning with a decrease in E[(ei − ej) 2] narrows the correctness gap that averaging can tolerate.
The system can be refined by using a k-nearest-neighbor graph constructed from penultimate features and holding it fixed, as this structure yield[s] lower NLL than random graphs in all three evaluated settings, indicating that which samples the graph connects is more important than indiscriminate smoothing.