Attention Sinks Induce Gradient Sinks: Massive Activations as Gradient Regulators in Transformers

arXiv:2603.17771 · cs.LG, cs.AI · Submitted 2026-03-18 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Attention Sinks Induce Gradient Sinks".

Jane: The gist The study shows that attention sinks and massive activations are related through backpropagation, where attention sinks induce gradient concentration,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: This next paper is titled "A Decision-Aware Evaluation Protocol for Uncertainty in Medical Image Segmentation." It seems like it's moving away from just looking at how well a model predicts an image to how good its decisions are for a clinical task.

Jane: That makes sense. So, instead of focusing on the model's uncertainty score itself, they want to evaluate the decisions it makes under specific constraints or budgets >

Lu: The authors are proposing an evaluation protocol where we assess what kinds of review decisions the model can make given explicit resource limitations >

Meng: I wonder how this translates to real-world deployment. If a radiologist is making a diagnosis, do they have a limited time or budget that affects the model's output?

Lalam: They are setting up a way to measure segmentation uncertainty by looking at what decisions that uncertainty allows them to make, rather than just standard scores >

Tom: So it’s about evaluating the utility of the model's output in a real workflow, not just its raw accuracy on some benchmark dataset >

Jane: And that sounds very practical. It suggests that for medical AI, how you measure performance needs to change based on the context of use >

The paper's summary: Tom: The summary points out that they are tackling the problem of evaluating medical image segmentation uncertainty by focusing on the review decisions enabled by those uncertainties under explicit budgets >

Jane: They’re essentially saying that you shouldn't just rely on calibration or error detection scores alone because those don't capture the full picture of what a model can do in a clinical setting >

Lu: They argue that standard metrics might not reflect how useful the segmentation is when there are real-world constraints involved, which is why they need a decision-aware protocol >

Meng: So if I'm shipping an AI for radiology, and I have to give the clinician a result quickly, that speed constraint should be factored into the evaluation of uncertainty >

Lalam: They are defining how we evaluate segmentation uncertainty by looking at the review decisions that are possible under explicit budgets, not just standard calibration or error detection scores >

Tom: And they define this protocol based on those budget constraints, which is a pretty concrete way to measure performance in a setting where efficiency matters >

Jane: It sounds like they’re providing a framework for how to judge whether a model's uncertainty is actually useful in practice, not just mathematically sound >

The paper's improvements: Tom: The authors propose this decision-aware protocol as the improvement over existing methods, which is that it directly addresses the issue of evaluating medical image segmentation uncertainty under explicit budgets >

Lu: They are showing that this protocol allows for a more nuanced evaluation because you can look at different decision possibilities based on those budgets >

Meng: So if we could use this to guide how we design the model, would it help us build something more efficient? Maybe a smaller model that is better at making quick decisions >

Jane: That’s a possibility. It suggests that the way you design the model should be informed by what kinds of decisions are important in the target application >

Conclusion: Tom: So to wrap up on this paper, we've discussed how this new evaluation protocol for medical image segmentation looks at uncertainty through a decision-aware lens rather than just calibration scores >

Jane: The key takeaway is that performance in a clinical setting depends on the decisions made under explicit budgets, which provides a framework for more useful evaluation in medical AI >

Lu: It shifts the focus to how we evaluate models based on what they can achieve when resources are constrained, which is a much more realistic way to assess performance >

Meng: For deployment, it means that if we can design models that make decisions under constraints, they will be better suited for those real-world scenarios >

Lalam: And I’m confident that this decision-aware evaluation protocol provides a robust way to measure uncertainty in segmentation tasks >

Tom: That’s all for this session. Thanks to Jane and Lu, Meng, and Lalam. We'll see you next time for more research updates.

Tsinghua University · Peking University

cs.LG, cs.AI

Submitted: 2026-03-18

Updated: 2026-10-19

Comments: 30 pages, 15 figures, 9 tables

Code: https://github.com/LARS-research/GradientSink

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 89/100

The gist: The gist The study shows that attention sinks and massive activations are related through backpropagation, where attention sinks induce gradient concentration, which massive activations act as

Key concepts

Gradient Sinks: These are identified as the backward-pass counterpart to attention sinks in pre-norm Transformers. They demonstrate that gradient pressure concentrates heavily at the first token, exhibiting a clear asymmetry where gradients spike at this location before dropping rapidly.
Massive Activations: These are not just forward causes of attention sinks but are understood as learned regulators. They absorb the sink-induced gradient pressure and help stabilize the flow of gradients through the residual stream in pre-norm architectures.
V-scale: V-scale is introduced as a value-path gradient valve designed to attenuate sink-induced gradients. It works by selectively modulating backpropagated signals along the value path, effectively suppressing massive activations while maintaining the essential attention sink structure.

Terminology

Transformer — 3× in this paper · explained in 2 episodes
Transformers — 2× in this paper · explained in 2 episodes
Transformer Model — 1× in this paper · explained in 2 episodes
Transformer Models — 1× in this paper · explained in 3 episodes

Summary

The gist The study shows that attention sinks and massive activations are related through backpropagation, where attention sinks induce gradient concentration, which massive activations act as adaptive regulators to stabilize gradient transport in pre-norm Transformers.

Gradient Sinks

1 Gradient sinks are identified as a backward-pass counterpart of attention sinks in pre-norm Transformers (Page 2).

2 They show that concentrated gradient pressure at the first token is widespread across models (Page 3).

3 The gradient sink ratio RlGS(s; Te) is defined as the ratio of token s's gradient norm to the mean over other early tokens, and it shows that the sink token consistently exhibits substantially larger gradients than other early tokens (Page 4).

4 The decomposition of gradients reveals a clear and highly structured asymmetry, where the gradient norm exhibits a pronounced spike at the first token across checkpoints, after which it drops rapidly and remains relatively flat over most of the sequence on key and value pathways (Page 4).

Massive Activations as Regulators

1 Massive activations can be understood as learned regulators that absorb sink-induced gradient pressure and help stable residual-stream gradient transport rather than just forward causes of attention sinks (Page 2).

2 RMSNorm makes activation scale an effective local factor to attenuate gradient pressure induced by attention sinks, thereby stabilizing gradient transport across the residual stream (Page 4).

3 The relationship between local gradient reshaping and input activation norms shows that large-activation points, almost all from token 0, are precisely where branch-local amplification becomes extreme, while the residual-stream gradient norms change only mildly for most layers (Page 5).

4 The Compress ratio indicates stronger attenuation at massive activation sites, revealing a strong negative relationship between activation scale and local gradient attenuation (Page 5).

V-scale Intervention

1 V-scale is introduced as a value-path gradient valve that attenuates sink-induced gradients (Page 5).

2 The intervention is designed to be strongest when the sink token has a small value-state norm, and it shows that if∥v∥22≪C for the sink token, the value-path backward signal is strongly attenuated (Page 3).

3 V-scale substantially suppresses massive activations while preserving attention sinks, illustrating that the two phenomena can be separated once gradient sinks are controlled (Page 7).

4 The intervention's effect is verified by showing that in the 0.3B V-scale model, the median RlGS(0) defined in (3) dropping from roughly 40.3 in the baseline to 5.6, which is the expected outcome of a partial intervention on the value-path gradient (Page 6).

Downstream Performance

1 Across both model scales, V-scale remains comparable to the baseline under bfloat16 and improves long-context retrieval (Page 9).

2 Table 7 shows that V-scale improves every listed precision/quantization and context-length combination, with especially large gains at 8192 tokens after YaRN extension (Page 23).

3 V-scale remains comparable to the baseline under bfloat16 and improves ARC-C, BoolQ, COPA, and OBQA and small drops on several other tasks (Page 20).

Theoretical Mechanism

1 The value-path backward signal is governed by Proposition 1, which states that the value-path backward signal satisfies∇vsL = A⊤G, where G is the upstream gradient at Y (Page 24).

2 Theorem 1 proves that sink structure provides a natural and quantitatively explicit route to value-path gradient concentration, showing that the dominant value-path spike is governed by sink statistics (Page 4).

3 Theorem 2 formalizes that a larger token-wise activation norm directly lowers the worst-case gain from upstream gradients to earlier layers, establishing massive activations as a gradient regulator rather than merely as pathological outliers (Page 4).

4 Corollary 2 shows that V-scale is a value-path gradient valve, where if∥v∥22≪C, then the value-path backward signal is strongly attenuated as∥∇vL∥2≲3∥v∥22Cgvˆ2 (Page 16).

This work identifies gradient sinks as a backward-pass counterpart of attention sinks in pre-norm Transformers and proposes V-scale, a value-path intervention that selectively modulates backpropagated gradients to suppress massive activations while preserving attention sinks. The findings suggest that massive activations emerge as an adaptive regulator responding to localized gradient pressure induced by attention sinks, which can be controlled by targeting the value path.

Limitations

1 This work has limitations because our study is restricted to Llama-like dense language models and it remains unclear whether the mechanism transfers to other settings like mixture-of-experts architectures (Page 9).

2 The analysis does not exclude other factors such as data distribution, optimizer, or numerical precision, from shaping AS and MA during training (Page 9).

References

[1] Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebron, and Sumit Sanghai. Gqa: Training generalized multi-query transformer models from multi-head checkpoints. In Empirical Methods in Natural Language Processing, 2023 (Page 10).

[2] Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. Systematic outliers in large language models. In International Conference on Learning Representations, 2025 (Page 14).

[3] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016 (Page 14).

[4] Federico Barbero, Alvaro Arroyo, Xiangming Gu, Christos Perivolaropoulos, Petar Velickovic, Razvan Pascanu, and Michael M. Bronstein. Why do llms attend to the first token? In Conference on Language Modeling, 2025 (Page 13).

[5] Yelysei Bondarenko, Markus Nagel, and Tijmen Blankevoort. Quantizable transformers: Removing outliers by helping attention heads do nothing. In Advances in Neural Information Processing Systems, 2023 (Page 13).

[6] Rui Bu, Haofeng Zhong, Wenzheng Chen, and Yangyan Li. Value-state gated attention for mitigating extreme-token phenomena in transformers. arXiv preprint arXiv:2510.09017, 2025 (Page 13).

[7] Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. GPT3.int8: 8-bit matrix multiplication for transformers at scale. In Advances in Neural Information Processing Systems, 2022 (Page 14).

[8] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Efficient finetuning of quantized LLMs. In Thirty-seventh Conference on Neural Information Processing Systems, 2023 (Page 14).

[9] Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. OPTQ: Accurate quantization for generative pre-trained transformers. In The Eleventh International Conference on Learning Representations, 2023 (Page 14).

[10] Zizhuo Fu, Wenxuan Zeng, Runsheng Wang, and Meng Li. Attention sink forges native moe in attention layers: Sink-aware training to address head collapse. arXiv preprint arXiv:2602.01203, 2026 (Page 14).

[11] Jorge Gallego-Feliciano, S. Aaron McClendon, Juan Morinelli, Stavros Zervoudakis, and Antonios Saravanos. Hidden dynamics of massive activations in transformer training. arXiv preprint arXiv:2508.03616, 2025 (Page 14).

[12] Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h et al. A framework for few-shot language model evaluation (Page 9).

[13] Aaron Grattafiori et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024 (Page 14).

[14] Xiangming Gu, Tianyu Pang, Chao Du, Qian Liu, Fengzhuo Zhang, Cunxiao Du, Ye Wang, and Min Lin. When attention sink emerges in language models: An empirical view. In International Conference on Learning Representations, 2025 (Page 14).

[15] Tianyu Guo et al. Active-dormant attention heads: Mechanistically demystifying extreme-token phenomena in llms. In Conference on Parsimony and Learning, 2025 (Page 13).

[16] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, 2016 (Page 14).

Improvements for AI systems

Attention sinks induce gradient sinks: Massive activations as Gradient Regulators in Transformers can be mitigated by identifying gradient sinks as a backward-pass counterpart to attention sinks and using V-scale, a value-path gradient valve, to attenuate sink-induced gradients while preserving attention sinks. This allows the AI system to suppress massive activations without sacrificing the beneficial sink structure.
The AI system can be trained more stably by applying V-scale, which is designed as a value-path gradient valve that attenuates sink-induced gradients. This intervention specifically targets the value path, showing that V-scale substantially suppresses massive activations while preserving attention sinks.
Quantization robustness can be enhanced for inference by using V-scale, which is shown to improve long-context retrieval accuracy across various quantization settings and provides positive gains in every listed precision, quantization method, and context length on downstream tasks.
Model architecture design can be informed by the finding that massive activations are learned regulators that absorb sink-induced gradient pressure, suggesting a design philosophy where architectural choices aim to preserve the benefits of sink structure while reducing its costs.

Abstract

Attention sinks and massive activations are recurring and closely related phenomena in Transformer models. Existing explanations have largely focused on the forward pass, yet in pre-norm Transformers, large residual-stream norms play only an indirect forward role because sublayers operate on normalized inputs. We study this relationship from the perspective of backpropagation. Empirically and theoretically, we show that under causal masking, attention sinks can induce pronounced gradient concentration, which we term gradient sinks. Since the RMSNorm Jacobian attenuates gradients roughly in inverse proportion to input norm, massive activations can be understood as adaptive regulators of this localized gradient pressure during training. This interpretation predicts that attenuating sink-induced gradients should weaken massive activations. We test this prediction with V-scale, a modification that adjusts backpropagated gradients on the value paths. In V-scale models and a backward-only control, attention sinks are preserved, whereas massive activations are suppressed. These results identify gradient sinks as a backward-pass counterpart of attention sinks, and massive activations as an adaptive RMSNorm-mediated response that attenuates the resulting localized training pressure. Our code is available at https://github.com/LARS-research/GradientSink.