Checking the Checkers: A Synthesis of Verifier Noise and Robustness in RLVR
cs.LG, cs.AI, cs.CL
Submitted: 2026-10-13
Updated: 2026-10-13
Comments: Accepted at the International Conference on Artificial Intelligence Centric Systems, Services, and Applications (AISSA 2026)
Code: https://github.com/reasoning-ai-lab/Checking-the-Checkers-RLVR
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Process Reinforcement through Implicit Rewards
- Where the Verifier Fails: A Category-Level Audit of Reward Signals in RLVR
- Auditing Reward Hackability in Code RL Training Environments
- One Token to Fool LLM-as-a-Judge
- Reward Hacking in Rubric-Based Reinforcement Learning
- RubricEval: A Rubric-Level Meta-Evaluation Benchmark for LLM Judges in Instruction Following
- An Imperfect Verifier is Good Enough: Learning with Noisy Rewards
- The Climb Carves Wisdom Deeper Than the Summit: On the Noisy Rewards in Learning to Reason
- Noisy Data is Destructive to Reinforcement Learning with Verifiable Rewards
- Delay, Plateau, or Collapse: Evaluating the Impact of Systematic Verification Error on RLVR
- Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges
- Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers
- From Accuracy to Robustness: A Study of Rule- and Model-based Verifiers in Mathematical Reasoning
- When the Reward Suite Is Leaky: A Preregistered Causal Contrast of Natural Verifier False Positives in RLVR
- LLMs Gaming Verifiers: RLVR can Lead to Reward Hacking
- AdvJudge-Zero: Binary Decision Flips in LLM-as-a-Judge via Adversarial Control Tokens
- Are Verifier Errors Independent Within a GRPO Group? Evidence from Qwen2.5 Rollouts
- Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panels
- From Rebound to Remedy: Understanding and Mitigating Reward Hacking via Representation Engineering
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks