Not All Errors Are Equal: Consequence-Aware Reasoning Compute Allocation
cs.AI
Submitted: 2026-06-03
Updated: 2026-10-01
License: http://creativecommons.org/licenses/by/4.0/
The gist: Test-time compute has emerged as an effective paradigm for improving large language model capability at inference time.
Terminology
Abstract
Test-time compute has emerged as an effective paradigm for improving large language model capability at inference time. Existing allocation strategies primarily prioritize tasks according to difficulty, uncertainty, or expected performance gain, implicitly treating prediction errors as equally costly. This assumption is often misaligned with real deployment, where failures can differ substantially in their downstream tasks. To address this limitation, this paper introduces consequence-aware test-time compute allocation by formulating a cost-weighted scheduling problem where the priority of a task is its failure consequence with the marginal gain of additional compute. In practice, however, marginal gain is difficult to predict before execution, so we propose a deployable scheduler that uses consequence as the routing signal. The scheduler predicts task consequence from pre-solution inputs and allocates the available premium compute to the corresponding top-ranked tasks. Experiments on the SWE bench Lite show that consequence provides information beyond task difficulty and can be predicted before solving. Under a fixed compute budget, consequence-aware routing achieves the best high-consequence task success, while overall accuracy remains competitive. A controlled within-model experiment further confirms the same advantage when only inference attempts are reallocated.
Sources
- FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
- Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generation
- RouteLLM: Learning to Route LLMs with Preference Data
- Adaptive Test-Time Compute Allocation via Learned Heuristics over Categorical Structure
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems