A Transport Geometry of Benign Overfitting

The gist: The study introduces a spectral-transport distance to quantify perturbations in covariance spectra and proves that functionals governing benign overfitting, such as effective ranks, are Lipschitz stable under this metric, leading to risk transfer theorems and empirical certification results.

Spectral-transport distance

The authors propose measuring spectral perturbations with the Wasserstein distance between spectral measures, which they call spectral transport <ref:2604.08625#pg4>. This distance is defined as the quadratic Wasserstein distance between normalized spectral counting measures, identified with the l2 distance of sorted spectra, and related to operator norms through Weyl’s inequality and the Hoffman-Wielandt inequality <ref:2604.08625#pg4>. On the real line, this coincides with the Euclidean distance between sorted eigenvalue sequences <ref:2604.08625#pg2>.

Stability of effective ranks

The paper proves that the tail functionals Tk, Sk and the two effective ranks rk, Rk that govern benign overfitting are Lipschitz stable in the spectral-transport distance <ref:2604.08625#pg4>. This stability is quantified by Proposition 3.3, which shows that if τ ≤ λk+1/2, then rk(Σ') - rk(Σ) ≤ 2τ/λk+1 (√p − k + rk(Σ)) <ref:2604.08625#pg6>. Furthermore, if in addition τ ≤ √Sk(Σ)/2, the stability of the second effective rank Rk is shown to be bounded by Rk(Σ') - Rk(Σ) ≤ c0 τ Ψk(Σ) <ref:2604.08625#pg6>.

Risk transfer and stability of benignity

The framework yields three main consequences: a risk transfer theorem, a stability theorem for the benign overfitting classification, and an empirical certification result <ref:2604.08625#pg4>. Theorem 4.1 states that under Assumption 4.1, the minimum norm interpolator trained on data with covariance Σ' satisfies (thetâ; Σ') ≤ B(Σ) + L(Σ) τ(Σ, Σ') with probability at least at least 1 − δ <ref:2604.08625#pg5>. Consequently, the benign overfitting classification is preserved along sequences with vanishing spectral-transport perturbations <ref:2604.08625#pg5>.

Empirical certification

The framework allows for empirical certification of benignity through the sample covariance operators <ref:2604.08625#pg4>. Corollary 4.3 shows that if √p εm ≤ δ̄(Σ), then the effective ranks computed from the empirical spectrum satisfy the two-sided bounds of Proposition 3.3 with τ ≤ √p εm, and in particular the empirical indices k∗(Σ̂ m)/n and n/Rk∗(Σ̂ m) certify benignity of the population model up to explicit additive errors <ref:2604.08625#pg9>.

Numerical validation

Numerical experiments with three spectral families confirm the theory <ref:2604.08625#pg4>. The ordering of the effective rank indices predicts the ordering of the empirical excess risks, and the observed risk change scales at a near Lipschitz rate in the Wasserstein distance between spectra <ref:2604.08625#pg4>. Table 1 reports that for Model C, as p/n grows, the flat tail absorbs an increasing share of the unit trace while the bias functional grows, which is predicted by Bbias while the variance indices remain small <ref:2604.08625#pg8>. Furthermore, Table 2 demonstrates that for Model B, even a large perturbation amplitude changes eigenvalues by up to 80% yet the risk moves by less than 2% of its base value <ref:2604.08625#pg7>.

Beyond alignment

The analysis also explores what happens when the eigenbasis is not aligned, where rotations of the eigenbasis are considered <ref:2604.08625#pg10>. The experiment of Section 5.5 quantifies what is at stake, since pure rotations leave the spectral-transport distance at zero yet move the risk more than aligned spectral perturbations of matched Frobenius norm <ref:2604.08625#pg12>. This suggests that any extension of Theorem 4.1 beyond alignment must include an eigenprojection term, for which the Davis-Kahan theorem [26] is the natural tool <ref:2604.08625#pg10>. The moderate size of the rotation effect indicates that the aligned theory captures the dominant part of the perturbation response in this model <ref:2604.08625#pg10>.

Limitations and extensions

Three limitations are identified: first, Theorem 4.1 assumes an aligned eigenbasis, and the misaligned case couples the spectral perturbation with a rotation of eigenprojections, which the Davis-Kahan theorem [26] controls only under eigenvalue gap conditions, and a sharp joint treatment is left open <ref:2604.08625#pg12>. Second, our experiments use Gaussian designs; universality of the phenomenon across sub-Gaussian and kernel designs is supported by [10, 25] but not proved here in full generality <ref:2604.08625#pg12>. Third, the constants in the admissibility threshold δ̄(Σ) are conservative, as Fig. 2 shows, and sharpening them, possibly through the precise asymptotics of [5, 9], is a natural next step <ref:2604.08625#pg12>. Extending the framework from linear and kernel models to trained representations would connect the theory more tightly to the practice documented in [1] <ref:2604.08625#pg12>.

Declaration of competing interest

The author declares that he has no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper <ref:2604.08625#pg14>.

Funding and Data availability

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors <ref:2604.08625#pg14>. No external datasets were used <ref:2604.08625#pg14>. The Python code that reproduces all simulations, tables and figures is available from the corresponding author upon reasonable request <ref:2604.08625#pg14>.

A. Proofs

The proofs are collected in Appendix A <ref:2604.08625#pg14>. The proof of Lemma 3.1 and of the sorted representation uses Weyl’s inequality and the Hoffman-Wielandt inequality <ref:2604.08625#pg14>. The proof of Proposition 3.3 involves applying Cauchy-Schwarz inequality to the tail difference vector against the all-ones vector of length p <ref:2604.08625#pg14>. The proof of Theorem 4.1 combines Steps 2–4, showing that B(Σ') ≤ B(Σ) + L(Σ)τ with L as stated <ref:2604.08625#pg14>. The proof of Corollary 4.3 uses Weyl’s inequality to bound the spectral distance and Proposition 3.3 applied with Σ' = Σ̂ √ m bounds the deviation of every empirical tail functional and effective rank by an explicit multiple of p εm <ref:2604.08625#pg14>. The proof of Theorem 4.2 shows that for large n, B(Σ(n)) → 0, and the hypothesis gives L(Σ(n)) τn → 0, so the excess risk converges to zero in probability <ref:2604.08625#pg14>. The proof of Corollary 4.3 applies to empirical kernel integral operators with spectral convergence results of [25] <ref:2604.08625#pg14>. The proof of Theorem 4.1 uses elementary Lipschitz estimates for the square root on these ranges <ref:2604.08625#pg14>. The proof of Corollary 4.3 relies on the operator norm bound being Corollary 2 of [24] applied to the sub-Gaussian covariates <ref:2604.08625#pg14>. The proof of Theorem 4.1 combines Steps 2–4, showing that B(Σ') ≤ B(Σ) + L(Σ)τ with L as stated <ref:2604.08625#pg14>. The proof of Corollary 4.3 uses Weyl’s inequality to bound the spectral distance and Proposition 3.3 applied with Σ' = Σ̂ √ m bounds the deviation of every empirical tail functional and effective rank by an explicit multiple of p εm <ref:2604.

1. Benign Overfitting Classification Stability: The improved system can reliably classify whether an interpolating estimator is benign even when its population covariance spectrum is perturbed by deformations of size defined by spectral transport distance, as proven in Theorem 4.2: the benign overfitting classification is preserved along sequences with vanishing spectral-transport perturbations.

2. Risk Transfer Under Spectral Perturbations: The system can predict the change in excess risk for a minimum norm interpolator under a perturbed covariance operator, bounding this change by an explicit Lipschitz constant times the spectral transport distance, as shown in Theorem 4.1: the excess risk of the minimum norm interpolator trained on data with covariance Σ′ satisfies (thetâ; Σ′) ≤ B(Σ) + L(Σ) tau(Σ, Σ′).

3. Empirical Certification of Benignity: The system can certify the benign nature of a population model using only an estimated sample covariance spectrum, provided the spectral-transport distance between the empirical and population spectra is sufficiently small, as detailed in Corollary 4.3: If √ p epsilonm ≤ deltā(Σ), then the effective ranks computed from the empirical spectrum certify benignity of the population model up to explicit additive errors.

4. Robustness Against Eigenbasis Misalignment: The system can provide a quantitative measure of performance degradation when the eigenbasis is not perfectly aligned, as demonstrated in Section 5.5: any extension of Theorem 4.1 beyond alignment must include an eigenprojection term, for which the Davis-Kahan theorem [26] is the natural tool.

Benign overfitting, the phenomenon that interpolating noisy data can be statistically harmless, is usually characterized through effective-rank conditions on the covariance spectrum. These conditions are fragile: a perturbation of vanishing relative operator and Frobenius norm can turn a benign covariance into one whose minimum-norm interpolator has non-vanishing excess risk. We argue that the natural geometry of benign overfitting is optimal transport of the log-eigenvalues, measured in units of the sample size n. In log-spectral coordinates, the deterministic equivalent of the risk becomes a functional of two measures smoothed by a logistic kernel centred at the interpolation threshold κ. We compute its exact first variation: the threshold moves by a sensitivity-weighted average of the log-eigenvalue displacements, and the influence of each eigenvalue decays exponentially with its log-distance to the threshold. We then derive global, dimension-free stability bounds. The variance functional is 8/(27n) -Lipschitz in the 1-Wasserstein distance between log-spectra, with a sharp constant, and is controlled more generally by a threshold-localized transport discrepancy; an analogous bound controls the bias. Consequently, any rank- k update of the covariance, of arbitrary norm, changes the variance functional by at most 3k/n, and benign overfitting survives every perturbation whose localized transport cost is o(n). The results extend to ridge regression and to infinite-dimensional trace-class covariances. Matching lower bounds show that the n-scaling is necessary and cannot be replaced by operator norms, Frobenius norms, or dimension-normalized spectral distances. Simulations, one-step feature learning in two-layer networks, and random features of real images support the theory and delineate its limits.