Hypotheses-Guided Self Distillation for Continual Personalization
cs.AI
Submitted: 2026-08-31
Updated: 2026-08-31
License: http://creativecommons.org/licenses/by/4.0/
The gist: As people increasingly interact with LLM assistants in daily life, continually adapting to individual preferences has become essential for effective long-term interactions.
Terminology
Abstract
As people increasingly interact with LLM assistants in daily life, continually adapting to individual preferences has become essential for effective long-term interactions. However, user preferences are rarely stated in full, and instead emerge through heterogeneous, latent, and noisy signals, with existing methods relying on raw interaction histories or costly reward-based optimization to manage personalization. We introduce HypReflect, a reliable, scalable framework for continual personalization that infers explicit, uncertainty-aware preference hypotheses from diverse user signals, reflectively refines them as new evidence accumulates, and incorporates the resulting user model through hypotheses-guided self-distillation. Experiments across three personalization settings: online personalization, multi-session interactions, and implicit behavioral signals, show that HypReflect outperforms a range of baselines, including raw-history and incremental-update methods. We further demonstrate strong generalization to unseen users and cross-domain settings, along with stability across context budgets, reusable hypotheses, and more focused personalization. These results suggest a step towards reliable and scalable continual personalization through explicit, revisable user preference hypotheses.
Sources
- Uncertainty-Aware Clarification in LLM Agents with Information Gain
- HyPerAlign: Interpretable Personalized LLM Alignment via Hypothesis Generation
- LLMs Get Lost In Multi-Turn Conversation
- A Survey of Personalized Large Language Models: Progress and Future Directions
- MultiSessionCollab: Learning User Preferences with Memory to Improve Long-Term Collaboration
- Learning to summarize user information for personalized reinforcement learning from human feedback
- Reaching Beyond the Mode: RL for Distributional Reasoning in Language Models
- User Modeling and User Profiling: A Comprehensive Survey
- Bayesian Teaching Enables Probabilistic Reasoning in Large Language Models
- Gemma 4 Technical Report
- Enhancing Personalized Multi-Turn Dialogue with Curiosity Reward
- Qwen3 Technical Report
- Personalization of Large Language Models: A Survey
- Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs
- Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
- Teaching Language Models to Evolve with Users: Dynamic Profile Modeling for Personalized Alignment
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems