Contrastive Branch Policy Optimization

arXiv:2608.24300 · cs.LG, cs.AI · Submitted 2026-08-25 · Read on arXiv

cs.LG, cs.AI

Submitted: 2026-08-25

Updated: 2026-08-25

Comments: 10 pages, 5 figures, 3 tables

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• GRPO — 16× in this paper · explained in 6 episodes
• Benchmark — 15× in this paper · explained in 2 episodes
• Reinforcement Learning — 14× in this paper · explained in 4 episodes
• Credit assignment — 12× in this paper · explained in 3 episodes
• Contrastive Branch Policy Optimization — 9× in this paper
• Ablation — 8× in this paper · explained in 1 episode
• Accuracy — 7× in this paper · explained in 1 episode
• Contrastive Branch Value — 4× in this paper
• Distillation — 4× in this paper · explained in 2 episodes
• Verification — 4× in this paper · explained in 2 episodes
• LLM-as-a-Judge — 3× in this paper · explained in 3 episodes
• Verifier — 3× in this paper · explained in 2 episodes
• Chain-of-Thought Prompting — 2× in this paper · explained in 2 episodes
• On-Policy Distillation — 2× in this paper · explained in 3 episodes
• Verifiable Rewards — 2× in this paper · explained in 2 episodes
• Ablation study — 1× in this paper · explained in 3 episodes
• Benchmarking — 1× in this paper · explained in 4 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• GRPO Objective — 1× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 8 episodes
• Local Optimization — 1× in this paper · explained in 2 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Reward Shaping — 1× in this paper · explained in 2 episodes
• Trace — 1× in this paper · explained in 2 episodes

Sources

• Off-Context GRPO: Learning to Reason on Hard Problems using Privileged Information
• Reasoning with Exploration: An Entropy Perspective
• Training Verifiers to Solve Math Word Problems
• Agentic Reinforced Policy Optimization
• Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning
• Agentic Entropy-Balanced Policy Optimization
• Group-in-Group Policy Optimization for LLM Agent Training
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Where Hindsight Credit Can Reside: A Signed-Capacity View of Token Updates in RLVR
• Measuring Mathematical Problem Solving With the MATH Dataset
• REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization
• Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs
• ToRL: Scaling Tool-Integrated RL
• Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
• Understanding Tool-Integrated Reasoning
• Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capability
• Proximal Policy Optimization Algorithms
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• ZeroSearch: Incentivize the Search Capability of LLMs without Searching
• Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks