Where Do LLM Ensemble Gains Come From? Decomposing Candidate Value, Diversity, and Aggregation

arXiv:2610.18786 · cs.CL, cs.AI · Submitted 2026-10-13 · Read on arXiv

cs.CL, cs.AI

Submitted: 2026-10-13

Updated: 2026-10-14

Comments: 14 pages, 6 figures

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

Aggregation — 1× in this paper

Sources

Quantifying Diversity of Thought: A Predictive Law of Weighted LLM Ensemble Lift
When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models
Universal Self-Consistency for Large Language Model Generation
Improving Factuality and Reasoning in Language Models through Multiagent Debate
Beyond Consensus: Trace-Level Synthesis in Mixture of Agents
Are Diversity Metrics Measuring Diversity? A Capability-Controlled Audit of Majority-Vote Gain in LLM Ensembles
LLM-as-a-Judge Scores Are Unreliable Optimization Signals in Closed-Loop Table Recognition
When LLMs Sit Above Diagnostic Tools: Unrealized Complementarity in Industrial Fault Diagnosis
Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?
Diverse Evidence, Better Forecasts: Multi-Agent Deliberation Under Information Asymmetry
Mixture-of-Agents Enhances Large Language Model Capabilities

Related papers