Radial-Angular Geometry for Reliable Update Diagnosis in Noisy-Label Learning

arXiv:2605.17429 · cs.LG, cs.CV · Submitted 2026-05-17 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Radial-Angular Geometry for Reliable Update Diagnosis in Noisy-Label Learning".

Jane: Here's the gist of the paper. The gist: The proposed Relative Geometric Conflict (RGC) method reformulates noisy-label reliability estimation as assessing the reliability of observed-label-induced parameter updates by comparing them against a reference update induced by an EMA teacher, distinguishing between large but aligned hard-clean updates and large conflicting updates caused by corrupted labels.

Tom: Problem Motivation Existing methods often estimate sample reliability from forward-space signals such as loss, confidence, or entropy which indicate whether a sample is difficult to predict but do not directly test whether its observed label induces a reliable parameter update <ref:2605.17429#pg2>.

Jane: This gap matters because hard clean samples and mislabeled samples can have similar loss while inducing different updates <ref:2605.17429#pg2>.

Tom: The sample-wise empirical Fisher trace gives a backward-space measure of update energy, factorizing into a prediction-residual term and a feature-sensitivity term, capturing information beyond scalar loss <ref:2605.17429#pg2>.

Paper summary: Jane: So, wrapping up the discussion on "Radial-Angular Geometry for Reliable Update Diagnosis in Noisy-Label Learning", what's the big picture here for folks working on AI systems?

Tom: It’s about shifting our focus from just looking at forward space signals like loss or confidence to actually diagnosing the parameter update behavior itself >Page two>.

Lu: The paper proposes that we use a combination of radial update energy and relative geometric conflict to score trust in noisy supervision >Page four>. This score helps us decide which samples to rely on more when training with noisy labels.

Meng: So, what does this mean for deployment? It suggests a continuous gate approach where we weight high-trust samples toward their observed labels and low-trust ones toward the reference target >Page five>. That avoids having to make a hard clean versus noisy split upfront.

Jane: Exactly. It’s about building a more reliable training loop where the trust in the label is assessed by how consistent it is with our understanding of what a good update looks like, not just by how high the loss number is >Page two>.

Tom: The title itself captures that idea well—Radial-Angular Geometry. It’s not just about magnitude; it’s about geometry in parameter space. It’s a way to look at the update as both a size and a direction >Page three> <ref:2605.17429#pg1>.

Lu: And the implication is that for noisy label learning, we can use this geometric conflict term to explicitly separate informative hard clean samples from those that are just corrupted by bad labels >Page two>. That's a concrete diagnostic tool.

Jane: So, it’s a new way to diagnose reliability by looking at the geometry of the gradients rather than just their scalar values >Page one> <ref:2605.17429#pg1>. It’s about understanding the update's relationship with our teacher model.

Conclusion: Tom: So we're wrapping up on "Radial-Angular Geometry for Reliable Update Diagnosis in Noisy-Label Learning" and its authors, Ningkang Peng and Jingyang Mao et al., which essentially redefines how we check if a noisy label is actually good.

Jane: It’s about taking the idea of reliability—like loss or confidence—and instead looking at the geometry of the parameter updates themselves >Page two>. So, they’re saying that how you update your model matters more than just how hard it was to guess the label in the first place.

Lu: Right. The core mechanism is this sample-wise empirical Fisher trace, which they break down into a prediction residual term and a feature sensitivity term >Page two>. That lets them see if that update energy is coming from a good signal or just noise >Page two>.

Meng: I'm wondering what the practical downside is here. If we're looking at gradients and geometry, how much overhead does this introduce for deployment on a massive model? What’s the real cost of calculating this relative conflict term you mentioned earlier?

Tom: That’s a fair point, Meng. The paper uses a lightweight surrogate—the classifier layer contribution tau last i —to keep the computation manageable >Page two>. It factorizes into prediction residual squared divided by feature sensitivity squared >Page two>. But they still need that reference gradient from an EMA teacher, which adds a bit of complexity to the training loop.

Jane: That’s where the relative geometric conflict comes in. It compares your observed label's gradient against what a smooth teacher model would suggest it should be doing >Page three>. They use a cosine similarity measure, C i = one - (g obs i, g ref i), to see if those two directions are pointing in the same general direction.

Lu: Exactly. That conflict term is what they use to separate a large update that's aligned with the teacher—which is likely a hard clean sample—from one that’s large but pointing in the wrong direction because of bad labels >Page three>. It’s about distinguishing useful updates from conflicting ones.

Meng: So, if I were deploying this, I'd be looking for that conflict signal to flag samples where the update magnitude is high but the angular mismatch is significant >Page four>. That should help me filter out those label corruption issues before they hit my production pipeline.

Tom: The final scoring uses both a radial mismatch term R tau i and an angular mismatch term C b epsilon i to create a trust score, R tau i = tau obs i + epsilon tau ref i + C b epsilon i, which they use in the final loss function >Page four>. It balances the magnitude and the direction of that update energy.

Jane: It means that instead of just trusting a sample because its loss is low, you're checking if its resulting parameter change is geometrically consistent with what a reliable model should be doing >Page four>. That’s a shift from simple forward-space difficulty to actual parameter space behavior.

Lu: The experiments on CIFAR and CIFAR100 show that this RGC method improves hard-clean preservation compared to just looking at the magnitude of the trace alone >Page four>. It's definitely picking up on that distinction between hard clean and noisy samples.

Meng: So, it’s not a magic fix, but it gives us a more nuanced filter for when to trust a label, which is what we need when scaling these models to real-world data where noise is everywhere. That’s the practical implication for me.

Ningkang Peng, Jingyang Mao, Xiaoqian Peng, Weiguang Qu, Yanhui Gu

Nanjing Normal University · Nanjing Normal University · Nanjing University of Chinese Medicine

cs.LG, cs.CV

Submitted: 2026-05-17

Updated: 2026-10-17

License: http://creativecommons.org/licenses/by-sa/4.0/

Importance score: 92/100

The gist: The gist: The proposed Relative Geometric Conflict (RGC) method reformulates noisy-label reliability estimation as assessing the reliability of observed-label-induced parameter updates by comparing

Key concepts

Forward Signals
These are easy metrics like loss or confidence used to judge how difficult a sample is to predict under the noisy label. The paper argues these signals are insufficient because hard clean samples and mislabeled samples can look similar in terms of forward difficulty but cause very different changes when training the model.
Empirical Fisher Trace
This is a backward-space measure of update energy that looks at how much the observed label influences the parameter updates. For the classifier layer, it captures both prediction error and how strongly that error affects feature sensitivity in the model's parameters, providing more detail than just scalar loss.
Relative Geometric Conflict (RGC)
RGC is a proposed method that diagnoses reliability by comparing two gradients: the observed-label gradient and a reference gradient from an Exponential Moving Average (EMA) teacher. A high conflict score indicates that the observed label causes an update direction significantly different from what the stable teacher suggests, helping to identify corrupted labels.
Reliability Scoring (Rτ_i)
This score combines two components: Rτ_i measures radial mismatch in update magnitude (how much the update size differs from the reference), while Cbϵ^i measures angular mismatch in direction. This combined metric provides a comprehensive assessment of whether an observed label-induced parameter change is trustworthy.

Terminology

Summary

The gist: The proposed Relative Geometric Conflict (RGC) method reformulates noisy-label reliability estimation as assessing the reliability of observed-label-induced parameter updates by comparing them against a reference update induced by an EMA teacher, distinguishing between large but aligned hard-clean updates and large conflicting updates caused by corrupted labels.

Problem Motivation

Existing methods often estimate sample reliability from forward-space signals such as loss, confidence, or entropy which indicate whether a sample is difficult to predict but do not directly test whether its observed label induces a reliable parameter update <ref:2605.17429#pg2>. This gap matters because hard clean samples and mislabeled samples can have similar loss while inducing different updates <ref:2605.17429#pg2>. The sample-wise empirical Fisher trace gives a backward-space measure of update energy, factorizing into a prediction-residual term and a feature-sensitivity term, capturing information beyond scalar loss <ref:2605.17429#pg2>. However, the trace is still a radial magnitude signal and cannot decide whether a large update is useful or harmful <ref:2605.17429#pg3>.

Methodology Components

The framework introduces three core ingredients to estimate reliability:

  1. EMA reference target: This provides the reference target for measuring whether the observed-label update is reliable, and it maintains a temporally smoothed teacher model <ref:2605.17429#pg4>. The teacher is not assumed to be an oracle but acts as a lightweight temporal ensemble of historical student models <ref:2605.17429#pg4>.

  2. Practical trace proxy: Since computing the full-model sample-wise empirical Fisher trace is expensive, a lightweight surrogate, the classifier-layer contribution, is used to measure update magnitude <ref:2605.17429#pg5>. This surrogate is defined as τ last i = ∥δi∥ 2 2∥hi∥ 2 2 <ref:2605.17429#pg5>.

  3. Relative Geometric Conflict (RGC): RGC compares the observed-label gradient with a reference gradient induced by an EMA teacher, quantifying inconsistency by Ci = 1 − cos(g obs i, g ref i) <ref:2605.17429#pg3>. This conflict term helps distinguish large but aligned hard-clean updates from large conflicting updates caused by corrupted labels <ref:2605.17429#pg4>.

Reliability Scoring and Optimization

The unreliability score is defined as Rτ i = log τ obs i + ϵ τ ref i + ϵ, si = βRbτ i + (1 − β)Cbϵ i <ref:2605.17429#pg4>. Here, Rτ i measures radial mismatch in update magnitude, while Cbϵ i measures angular mismatch in update direction <ref:2605.17429#pg4>. The final trust-weighted objective is LRGC = 1/N Σ [wil(fθ(xi), y˜i) + λ(1 − wi)l(fθ(xi), qi)] <ref:2605.17429#pg5>. This continuous gate avoids a hard clean/noisy split by weighting high-trust samples toward their observed labels and low-trust samples toward the reference target <ref:2605.17429#pg5>.

Experimental Validation

Experiments on CIFAR-10 and CIFAR-100, as well as real-world datasets, show that RGC is the top entry among evaluated methods <ref:2605.17429#pg8>. The diagnostic analysis confirms that forward-space signals are weak under matched difficulty, trace improves detection but remains vulnerable to hard-clean false positives, and relative geometric conflict helps separate these cases along the conflict dimension <ref:2605.17429#pg9>. Ablation studies demonstrate that full RGC model has the highest or tied-highest entries in reliability metrics <ref:2605.17429#pg9>. The performance gains are consistent with the intended role of RGC, showing improvements over magnitude-only trace criteria when the reference is reliable <ref:2605.17429#pg9>.

Conclusion

RGC successfully combines radial update energy with observed–reference conflict to score trust in noisy supervision <ref:2605.17429#pg4>. The method improves over magnitude-only trace criteria when the reference is reliable, with gains shrinking when the teacher can drift <ref:2605.17429#pg9>. Code is available at https://anonymous.4open.science/r/AAA-BF68/.

--- Page 1 ---

Radial-Angular Geometry for Reliable Update Diagnosis in Noisy-Label Learning

Diagnosis in Noisy-Label Learning

Ningkang Peng, Jingyang Mao, Xiaoqian Peng, Weiguang Qu, Yanhui Gu <ref:2605.17429#pg2>

Abstract

Noisy-label methods often estimate sample reliability from forward-space signals such as loss, confidence, or entropy <ref:2605.17429#pg2>. These signals indicate whether a sample is difficult to predict, but they do not directly test whether its observed label induces a reliable parameter update <ref:2605.17429#pg2>. This gap matters because hard clean samples and mislabeled samples can have similar loss while inducing different updates <ref:2605.17429#pg2>. We recast reliability estimation as diagnosis of the observed-label update <ref:2605.17429#pg2>. The sample-wise empirical Fisher trace gives a backward-space measure of update energy <ref:2605.17429#pg2>. For the classifier layer, it factorizes into a prediction-residual term and a feature-sensitivity term, so it captures information beyond scalar loss <ref:2605.17429#pg2>. Trace, however, is still a radial magnitude signal and cannot decide whether a large update is useful or harmful <ref:2605.17429#pg3>. We therefore propose Relative Geometric Conflict (RGC), which compares the observed-label gradient with a reference gradient induced by an EMA teacher <ref:2605.17429#pg4>. The conflict term helps distinguish large but aligned hard-clean updates from large conflicting updates caused by corrupted labels <ref:2605.17429#pg4>. Across synthetic and real-world noisy-label benchmarks, RGC improves hard-clean preservation and accuracy under our evaluation protocol <ref:2605.17429#pg4>.

1 Introduction

Learning with noisy labels (LNL) can be viewed as a problem of unreliable supervision [2605.17429 pg1]. Most existing methods estimate sample reliability from forward-space quantities such as loss, confidence, or entropy [2605.17429 pg1]. These signals are convenient because they are available during training, but they primarily answer how difficult the current prediction is under the observed label <ref:2605.17429#pg2>. They leave open the optimization question of whether the update induced by that label is trustworthy <ref:2605.17429#pg2>. Hard clean samples and mislabeled samples can therefore overlap in forward space, since both may exhibit high loss or low confidence, especially under instance-dependent or feature-dependent noise [2605.17429 pg1].

This observation motivates us to shift the reliability criterion from forward difficulty to parameterspace update behavior <ref:2605.17429#pg2>. For a sample with observed label y˜i, the actual training effect is determined by the gradient g obs i = ∇θl(fθ(xi), y˜i) <ref:2605.17429#pg2>. The sample-wise empirical Fisher trace, which reduces to the squared gradient norm for gig T i, gives a backward-space diagnostic of this effect [2605.17429 pg1]. For the classifier layer, the trace factorizes as τ last i = ∥pi − ey˜i∥ squared / 2∥hi∥ 2 2 <ref:2605.17429#pg2>, showing that it captures not only the prediction residual but also how strongly this residual is transmitted into parameter updates through Preprint. arXiv:2605.17429v1 [cs.LG] 17 May 2026

--- Page 2 ---

the feature representation <ref:2605.17429#pg2>. Thus, trace provides information beyond the scalar forward loss and can distinguish samples that have similar loss but different update energy <ref:2605.17429#pg2>.

Improvements for AI systems

  1. Improved reliability estimation by shifting from forward-space signals like loss to backward-space diagnostic of update energy using Fisher trace to capture information beyond scalar loss. This allows the system to distinguish between hard clean samples and mislabeled samples that may have similar forward losses but induce different parameter updates, addressing the gap where hard clean samples and mislabeled samples can overlap in forward space.

  2. Implementation of Relative Geometric Conflict (RGC) for directional reliability testing. RGC compares the observed-label gradient with a reference gradient induced by an EMA teacher to distinguish large but aligned hard-clean updates from large conflicting updates caused by corrupted labels, thereby improving hard clean preservation and accuracy under noisy label benchmarks.

  3. Development of a trust-weighted optimization objective (LRGC). This objective converts the reliability score into a sample weight, allowing the system to train with high-trust samples weighted toward their observed labels and low-trust samples toward the reference target, creating a continuous gate that avoids an explicit hard clean/noisy split.

  4. Use of teacher-based reference targets for stable directionality. The EMA teacher provides a reference gradient for testing the geometric reliability of the observed-label update, ensuring that reliability assessment is based on whether the observed-label update is directionally aligned with the reference rather than an absolute test of difficulty or magnitude.

Abstract

Noisy-label methods often estimate sample reliability from forward-space signals such as loss, confidence, or entropy. These signals indicate whether a sample is difficult to predict, but they do not directly test whether its observed label induces a reliable parameter update. This gap matters because hard clean samples and mislabeled samples can have similar loss while inducing different updates. We recast reliability estimation as diagnosis of the observed-label update. The sample-wise empirical Fisher trace gives a backward-space measure of update energy: for the classifier layer, it factorizes into a prediction-residual term and a feature-sensitivity term, so it captures information beyond scalar loss. Trace, however, is still a radial magnitude signal and cannot decide whether a large update is useful or harmful. We therefore propose Relative Geometric Conflict (RGC), which compares the observed-label gradient with a reference gradient induced by an EMA teacher. The conflict term helps distinguish large but aligned hard-clean updates from large conflicting updates caused by corrupted labels. Across synthetic and real-world noisy-label benchmarks, RGC improves hard-clean preservation and accuracy under our evaluation protocol.

Sources

Related papers