Memorization and Malign Generalization in Conditional Diffusion Models with Random Features

arXiv:2610.11288 · cs.LG · Submitted 2026-10-08 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Memorization and Malign Generalization in Conditional Diffusion Models with Random Features".

Jane: The gist: In this work, we analyze a random-feature conditional score model in the high-dimensional proportional limit, deriving asymptotic expressions for training and test losses;

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So, to recap, this paper tackles the question of whether conditional diffusion models are just memorizing or actually learning when they produce samples under certain conditions. They analyze a random-feature conditional score model in a high-dimensional proportional limit to get the asymptotic expressions for training and test losses.

Jane: The main claim is that they show that in the overparameterized regime, increasing model width helps predict the condition-dependent mean better while simultaneously making the within-condition prediction variance smaller.

Lu: This specific trade-off they describe is what they term "malign generalization," which is a key finding when looking at how these models perform under certain conditions.

Tom: And on top of that, analyzing the training loss, they found that more informative conditions lead to memorization of training samples even at smaller model widths.

Meng: So it’s not just about brute force width; it’s about the quality and informativeness of the condition itself dictating how much memorization is needed.

Jane: They support these theoretical findings with experiments using U-Net architectures on realistic data, showing that a higher SNR makes memorization happen at smaller widths, and the variance keeps decreasing with width.

Lu: The paper also discusses how they decompose the test loss into conditional-mean error and residual error to quantify these two distinct prediction aspects.

Tom: So, it’s a deep dive into the mechanics of when these models memorize versus when they generalize well under specific constraints, which is important because recent work has mostly focused on unconditional settings.

Jane: It pushes us to think about how we can control the trade-off between accuracy and diversity in these conditional diffusion models.

Conclusion: Tom: So, wrapping up the discussion on "Memorization and Malign Generalization in Conditional Diffusion Models with Random Features," the authors introduce a tractable data model and derive asymptotic expressions for the test loss and training loss to map out memorization versus generalization.

Jane: The main results show that wider overparameterized models improve prediction of the condition-dependent mean while reducing within-condition prediction variance, which they label as malign generalization.

Lu: Furthermore, analyzing the training loss showed that more informative conditions cause memorization of training samples at smaller widths.

Meng: This suggests that for practical AI systems, we might not need massive parameter counts if our conditioning signals are strong enough to drive memorization at a smaller scale.

Tom: The authors also found that this is supported by experiments with U-Net architectures on realistic data, confirming their theoretical findings about higher SNR leading to memorization at smaller widths.

Jane: Ultimately, the paper suggests that we need to look into the mechanisms specific to conditional diffusion models regarding how conditioning influences those behaviors.

Lu: The authors also point out that they rely on assumptions like the Gaussian equivalence principle for nonlinear random-feature matrices, which is a limitation they admit when discussing future work.

Tom: It’s a solid piece of theory that helps us understand the limits of what these conditional diffusion models are capable of doing in terms of memorization and generalization.

Gwangho Kim, Sungyoon Lee

Hanyang University

cs.LG

Submitted: 2026-10-08

Updated: 2026-10-08

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 92/100

The gist: The gist: In this work, we analyze a random-feature conditional score model in the high-dimensional proportional limit, deriving asymptotic expressions for training and test losses; by decomposing

Key concepts

Random-Feature Conditional Score Model: This is a mathematical model used to describe conditional diffusion models. It involves defining specific embeddings and projectors that allow the condition to explain some variance in the data while leaving other parts unexplained, helping analyze how conditions influence learning.
Malign Generalization: This phenomenon occurs in overparameterized models where increasing model width improves the accuracy of predicting the true conditional mean. However, this improvement comes at a cost: the prediction variance within that condition decreases below what is expected from the true denoiser, leading to less diversity.
Memorization at Smaller Widths: The analysis of training loss reveals that when a condition is highly informative, the model starts memorizing specific training samples even when it has relatively small width. This suggests that strong conditioning can lead to overfitting or memorization earlier in the training process.
High-Dimensional Proportional Limit: This mathematical limit simplifies the analysis by considering models with very high dimensions and large sample sizes. It allows researchers to derive asymptotic expressions for training and test losses, providing theoretical insights into model behavior under these extreme conditions.

Terminology

Diffusion Model — 1× in this paper · explained in 10 episodes
Diffusion Models — 1× in this paper · explained in 9 episodes
Generalization — 1× in this paper · explained in 13 episodes

Summary

The gist: In this work, we analyze a random-feature conditional score model in the high-dimensional proportional limit, deriving asymptotic expressions for training and test losses; by decomposing the test loss, we show that in the overparameterized regime, increasing model width improves prediction of the condition-dependent mean while reducing within-condition prediction variance, a phenomenon we term “malign generalization”; furthermore, analyzing the training loss reveals that more informative conditions lead to memorization of training samples at smaller widths; these theoretical findings are supported by experiments with U-Net architectures on realistic data

Theoretical Framework and Data Model

The paper introduces a tractable data model (4) for conditional diffusion models, which captures key phenomena in realistic settings This model involves defining orthonormal condition embeddings U⋆ and two projectors Π and Π⊥, where the sample x is constructed as x = By + Σ1/2 ξ, with B = cU⋆, Σ = (1 − c 2)Π + Π⊥, 0 ≤ c ≤ 1 This structure allows the condition to explain variance along specific directions while leaving residual variance and unobserved direction variance The signal-to-noise ratio of the condition is defined as SNR(ψy, c):= lim tr(BB⊤)tr Σ = c 2ψy 1 − c 2ψy, which varies depending on the special case studied

Analysis of Test Loss and Malign Generalization

The test loss admits an orthogonal decomposition into the conditional-mean error (ME) and residual error (RE), where ME quantifies how accurately the denoiser captures the true conditional mean when averaged over within-condition variation, and RE evaluates how faithfully the sample-wise fluctuations of xA match the true residual x − By The paper derives asymptotic expressions for these quantities in the high-dimensional proportional regime, showing that in the overparameterized regime wider models predict the conditional mean more accurately while their prediction variance falls below that of the true denoiser, which is termed “malign generalization” Specifically, Remark 3.1 states that ME and PV describe single-step denoising properties at a fixed time t rather than the full multi-step trajectories of the reverse process

Analysis of Training Loss and Memorization

The analysis of the training loss reveals that more informative conditions lead to memorization of training samples at smaller widths The paper shows that in the proportional high-dimensional limit, Rtrain(r) describes how the training loss depends on width and on the condition, where Rtrain(r) = M h a 2 t ρ s ht + a 2 t ρ s i For large widths and large sample ratios, this leads to full memorization as r → ∞

Key Findings and Experimental Validation

The theoretical results are supported by experiments with U-Net architectures on realistic data The paper shows that in the full reverse-process generation with random features and U-Nets, a higher SNR makes memorization set in at smaller widths; the variance falls below the true conditional variance υ2Σ and keeps decreasing with width; and the conditional error peaks near ψp = ψn and decreases beyond it The results call for further investigation into memorization and generalization mechanisms specific to conditional diffusion models

Limitations and Future Directions

The analysis relies on assumptions such as the Gaussian equivalence principle (GEP) for nonlinear random-feature matrices, which is used to replace nonlinear functions with their Gaussian equivalents Future work is suggested in several directions, including investigating how conditioning mechanisms like cross-attention influence memorization and generalization Extending the theory to anisotropic marginals and non-Gaussian data remains an open problem The paper also notes that practical models typically inject the condition via cross-attention or adaptive normalization, which is a direction for future work The analysis does not address cases where training pairs are drawn with sampling biases such as duplicated images

Conclusion

The introduction of a tractable data model and the derivation of proportional limits for the test loss and training loss provide insights into memorization and generalization in conditional diffusion models The findings suggest that wider overparameterized models become more accurate but less diverse, while more informative conditions drive memorization at smaller widths These results call for further investigation into the mechanisms specific to conditional diffusion models

Improvements for AI systems

Improve conditional diffusion model training by utilizing informative conditions to mitigate memorization; a more informative condition lowers it at every width, so memorization sets in at smaller widths.
Develop overparameterized models that exhibit malign generalization by improving the prediction of the condition-dependent mean while reducing within-condition prediction variance.
Design a model where training loss is minimized for more informative conditions, leading to memorization at smaller widths, as described by the finding that more informative conditions lead to memorization of training samples paired with training conditions at smaller widths.
Implement a score-based denoiser whose conditional mean error (ME) and prediction variance (PV) exhibit behavior where the conditional-mean error peaks near the onset of memorization and decreases beyond it, as shown in Figure 6.
Create a system that leverages the decomposition of test loss into conditional-mean and residual errors to separate accurate conditional mean prediction from faithful sample-wise fluctuation matching, allowing for targeted regularization.