Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility
cs.LG, cs.AI
Submitted: 2026-08-04
Updated: 2026-08-31
Code: https://github.com/bespokelabsai/curator
Project page: https://mohsenhariri.github.io/scorio/ttshttps://huggingface.co/datasets/harimo/scorio
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language models can solve harder reasoning problems with more inference-time compute.
Terminology
Abstract
Large language models can solve harder reasoning problems with more inference-time compute. The term "test-time scaling," however, covers several inference algorithms: extending deliberation along one trajectory, sampling completed candidates and aggregating them by voting or verification, and searching over partial states. These algorithms differ in statistical structure, compute requirements, and failure modes. Treating them as interchangeable under a scalar "budget," or reporting accuracy without specifying the inference protocol, makes results difficult to compare across studies. We study test-time scaling along three axes. First, we formalize it as budgeted inference over the implicit prefix tree of an autoregressive model and distinguish single-trajectory sequential scaling, leaf-level scaling with terminal reduction, and prefix-level scaling. Second, we treat the full inference system as the evaluated object and separate end-to-end performance from candidate-bank diagnostics. We introduce an evaluation profile whose coordinates and simple functionals recover or bound common repeated-sampling metrics, and require compute accounting and uncertainty estimates that match the protocol. Third, we distinguish exact replay from distributional reproducibility and state the requirements for each. We also organize open-weight reasoning models by model-side and interface mechanisms. Our empirical study covers broad knowledge, symbolic reasoning, and competition mathematics, and we publicly release 1,403,520 sampled model attempts. The project website is available at https://mohsenhariri.github.io/scorio/tts. The released datasets are Trace (https://huggingface.co/datasets/harimo/scorio-trace), Lite (https://huggingface.co/datasets/harimo/scorio-lite), Math (https://huggingface.co/buckets/harimo/scorio-math), and SuperGPQA (https://huggingface.co/buckets/harimo/scorio-gpqa).
Sources
- Phi-4-reasoning Technical Report
- Phi-4 Technical Report
- OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critique
- OpenCodeReasoning: Advancing Data Distillation for Competitive Coding
- Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs
- NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model
- We need to talk about random seeds
- Towards Reproducible LLM Evaluation: Quantifying Uncertainty in LLM Benchmark Scores
- Evaluating Large Language Models Trained on Code
- Universal Self-Consistency for Large Language Model Generation
- Faster Minimum Bayes Risk Decoding with Confidence-based Pruning
- Training Verifiers to Solve Math Word Problems
- Process Reinforcement through Implicit Rewards
- Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs
- OpenThoughts: Data Recipes for Reasoning Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Truncation Sampling as Language Model Desmoothing
- Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignment
- Hyperparameter-Free Approach for Faster Minimum Bayes Risk Decoding
- LLM-as-a-Verifier: A General-Purpose Verification Framework
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks