Review Rarely Beats the Stronger Model: Separating Verification from Capability Transfer in LLM Pipelines
cs.CL, cs.AI
Submitted: 2026-10-11
Updated: 2026-10-14
Comments: 22 pages, 7 figures, 12 tables. Code: https://github.com/Kamilaabb/verification-or-capability-transfer
Code: https://github.com/huggingface/math-verify
License: http://creativecommons.org/licenses/by/4.0/
The gist: Qwen3-8B reviewing Llama-3.1-8B in math adds 33.2 points, yet Qwen3-8B alone is 6.4 points more accurate than the result.
Terminology
Abstract
Qwen3-8B reviewing Llama-3.1-8B in math adds 33.2 points, yet Qwen3-8B alone is 6.4 points more accurate than the result. Reviewing another model's answer is usually judged by its gain over the generator. We show that this confuses verification, which adds accuracy beyond either model alone, with capability transfer, where a stronger reviewer supplies answers it could have given on its own. Measured against the stronger model alone, across 48 pairs of 12 open-weight models (3B to 72B) in math, knowledge, code and reasoning, review beats it in only 9 of 96 review directions, and does worse the larger the capability gap. The reason is anchoring: the reviewer mostly follows the generator, discarding its own correct answers. Review costs 2.6 to 2.8 times as much as the stronger model alone, while a selector using the same two calls is 2.7 to 4.2 points more accurate than review and cheaper. Studies of verification should report the stronger model alone, not only the gain over the generator.
Sources
- Seed-Coder: Let the Code Model Curate Data for Itself
- Commit-first LLM judging inherits the judge's own errors
- The Llama 3 Herd of Models
- Qwen2.5-Coder Technical Report
- JEV-as-a-Judge: Accept When Confident, Escalate When Unsure
- When to Screen, When to Bypass: LLM-Judges in Resource-Scarce AI-Human Workflow
- When Does Verification Pay Off? A Closer Look at LLMs as Solution Verifiers
- Easier to Mislead Than to Correct: Harmful and Beneficial Revision in LLM Conformity
- Cross-Model LLM Code Review: Should you use Claude to review Codex or vice versa?
- Qwen3 Technical Report
- Qwen2.5 Technical Report
- Precise but Uncoupled: Reviewer Precision Does Not Guarantee Critique Uptake in Multi-Agent Math Reasoning
- More Convincing, Not More Correct: Self-Play Reward Hacking of Reference-Free LLM Judges
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering