A Gradient Flow Perspective on Minimum MMD Estimation
Sophia Seulkee Kang, Louis Sharrock, Xiaoyuan Cheng, François-Xavier Briol, Zonghao Chen
cs.LG, stat.ML
Submitted: 2026-07-04
Code: https://github.com/seulkang0518/MDF_AL
License: http://creativecommons.org/licenses/by/4.0/
The gist: Minimum maximum mean discrepancy (MMD) estimation has emerged as a robust and likelihood-free alternative to maximum likelihood estimation for parameter estimation.
Terminology
Abstract
Minimum maximum mean discrepancy (MMD) estimation has emerged as a robust and likelihood-free alternative to maximum likelihood estimation for parameter estimation. Yet, despite its practical success, the associated optimization problem remains poorly understood, with theoretical guarantees for existing algorithms hinging on convexity assumptions that rarely hold in practice. We address this gap by proposing a preconditioned gradient descent (PGD) scheme, establishing its asymptotic global convergence under explicit gradient-dominance and projection-residual conditions. Our approach is inspired by recent progress on MMD gradient flows, a nonparametric descent scheme on the space of probability measures. We provide extensive empirical evidence that our PGD scheme outperforms standard gradient descent across a range of challenging parameter estimation and composite hypothesis testing problems.
Sources
- On the Convergence of SGD with Biased Gradients
- Estimation of time series by Maximum Mean Discrepancy
- Statistical Inference for Generative Models with Maximum Mean Discrepancy
- Total robustness in Bayesian Nonlinear Regression
- A Deterministic Sampling Method via Maximum Mean Discrepancy Flow with Adaptive Kernel
- Parametric MMD Estimation with Missing Values: Robustness to Missingness and Data Model Misspecification
- On the Wasserstein Gradient Flow Interpretation of Drifting Models
- Minimum Distance Summaries for Robust Neural Posterior Estimation
- Minimum Wasserstein distance estimator under covariate shift: closed-form, super-efficiency and irregularity
- Nonparametric Instrumental Variable Regression with Observed Covariates
- Sobolev Regularized MMD Gradient Flow
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks