Learning omega-Powered Conditional Distributions with Importance-Weighted Flow Matching
cs.LG, cs.CV
Submitted: 2026-10-12
Updated: 2026-10-12
Comments: Code available at https://bitbucket.org/dmmlgeneva/iw-cfm
Journal ref: Transactions on Machine Learning Research (TMLR), 2026
Code: https://github.com/zhengqigao/REG
License: http://creativecommons.org/licenses/by/4.0/
The gist: Classifier-Free Guidance (CFG) with guidance scale γ is commonly motivated by the ω-powered conditional distribution obtained by setting ω=γ, but CFG does not, in general, sample from this
Terminology
Abstract
Classifier-Free Guidance (CFG) with guidance scale γ is commonly motivated by the ω-powered conditional distribution obtained by setting ω=γ, but CFG does not, in general, sample from this distribution exactly. We study whether the powered target can instead be learned explicitly. We introduce Importance-Weighted Conditional Flow Matching (IW-CFM), which reweights conditional flow-matching examples using density ratios obtained from frozen conditional and unconditional flow models. With exact normalized weights, the resulting objective has the same gradient as flow matching on the powered target; in practice, we use self-normalized estimation and a regularized empirical approximation for high-dimensional data. On synthetic distributions, IW-CFM recovers analytical powered targets for which numerically matched CFG, γ=ω, exhibits a visible mismatch. On ImageNet, an amortized lightweight corrector achieves better alignment than fixed- and matched-guidance CFG with the corresponding weighted and filtered reference distributions. We additionally present an exploratory CLIP analysis of powered-density rankings on real ImageNet data. Aggregate shifts across selected prompt-specific contrasts coexist with heterogeneous classwise responses, including nonmonotonic trajectories. These results provide a practical route for studying and approximating model-defined powered conditional distributions.
Sources
- Flow Matching in Latent Space
- Learn to Guide Your Diffusion Model
- Guided Flows for Generative Modeling and Decision Making
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks