RM-OPD: On-Policy Distillation for Generative Reward Models
cs.LG, cs.CL
Submitted: 2026-10-14
Updated: 2026-10-15
Comments: 34 pages, 4 figures; includes references and appendices
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Critique-out-Loud Reward Models
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- Flow-OPD: On-Policy Distillation for Flow Matching Models
- GLM-5: from Vibe Coding to Agentic Engineering
- Reinforcement Learning via Self-Distillation
- Draft-OPD: On-Policy Distillation for Speculative Draft Models
- Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- Generative Reward Models
- Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Reward Modeling from Natural Language Human Feedback
- MiMo-V2-Flash Technical Report
- Qwen2.5 Technical Report
- Self-Distilled RLVR
- Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation
- On-Policy Context Distillation for Language Models
- Beyond Scalar Reward Model: Learning Generative Judge from Preference Data
- RewardAnything: Generalizable Principle-Following Reward Models
- Self-Evaluation Is Already There: Eliciting Latent Judge Calibration in Base LLMs with Minimal Data
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks