Read the Room Before Your Model Generates
cs.AI, cs.CL
Submitted: 2026-10-13
Updated: 2026-10-19
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language models (LLMs) generally rely on sampling temperature to regulate the amount of randomness in their outputs.
Abstract
Large language models (LLMs) generally rely on sampling temperature to regulate the amount of randomness in their outputs. Mainstream methods for selecting temperature typically involve fixed, heuristic global settings and context or token-level calibration, rather than a geometrically interpretable representation of the communicative scenario. In this work, we introduce a simple and interpretable temperature selection framework, the Scenario-Actor, which adapts decoding to the inferred situational characteristics. Scenario-Actor extracts scenario-relevant structures from Transformer hidden states and determines the temperature from scenario embedding, a structured geometric representation of the communicative setting. Empirical results show that on Llama-3.1-8B-Instruct, Scenario-Actor achieves strong goal-completion performance in interactive social scenarios, significant gains on open-ended instruction following, and competitive performance on multi-turn conversation compared to multiple baselines.
Sources
- KL-Divergence Guided Temperature Sampling
- Training Verifiers to Solve Math Word Problems
- Adaptive Decoding via Latent Preference Optimization
- The Llama 3 Herd of Models
- GPT-4 Technical Report
- GPT-4o System Card
- Contextual Temperature for Language Modeling
- Re2: A Consistency-ensured Dataset for Full-stage Peer Review and Multi-turn Rebuttal Discussions
- EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Sampling
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems