Learning Ordinal Probabilistic Reward from Preferences
cs.CL
Submitted: 2026-02-13
Updated: 2026-10-11
Comments: 28 pages, 5 figures, ICLR 2026
Code: https://github.com/ritzz-ai/OPRM
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- GPT-4 Technical Report
- Critique-out-Loud Reward Models
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- InternLM2 Technical Report
- JudgeLRM: Large Reasoning Models as a Judge
- RM-R1: Reward Modeling as Reasoning
- UltraFeedback: Boosting Language Models with Scaled AI Feedback
- Enhancing Chat Language Models by Scaling High-quality Instructional Conversations
- Quantile Regression for Distributional Reward Models in RLHF
- How to Evaluate Reward Models for RLHF
- The Llama 3 Herd of Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Reward Reasoning Model
- Won't Get Fooled Again: Answering Questions with False Premises
- GPT-4o System Card
- Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
- RewardBench: Evaluating Reward Models for Language Modeling
- Implicit Actor Critic Coupling via a Supervised Learning Framework for RLVR
- TruthfulQA: Measuring How Models Mimic Human Falsehoods
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering