Does self-distillation improve reasoning?

arXiv:2610.18613 · cs.LG, cs.AI, cs.CL · Submitted 2026-10-13 · Read on arXiv

cs.LG, cs.AI, cs.CL

Submitted: 2026-10-13

Updated: 2026-10-13

Comments: 33 pages, 11 figures, 24 tables

Code: https://github.com/sleepypandu/does-sd-improve-reasoning

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Distillation — 1× in this paper · explained in 2 episodes
• Self-Distillation — 1× in this paper · explained in 7 episodes

Sources

• Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs
• Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
• Aligning Language Models from User Interactions
• Process Reinforcement through Implicit Rewards
• MiniLLM: On-Policy Distillation of Large Language Models
• OpenThoughts: Data Recipes for Reasoning Models
• Privileged, but Biased: How PI-Conditioned Teachers Break Self-Distillation
• Distilling the Knowledge in a Neural Network
• Reinforcement Learning via Self-Distillation
• Privileged Solutions or Context-Induced Teacher Behavior? Dissecting On-Policy Self-Distillation
• LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
• Rethinking On-Policy Self-Distillation for Thinking Models
• Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?
• Kimi K3: Open Frontier Intelligence
• Tulu 3: Pushing Frontiers in Open Language Model Post-Training
• MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training
• An Empirical Model of Large-Batch Training
• Privileged Likelihood Is Not Automatically Value: Three Checks for Token Credit in On-Policy Self-Distillation
• Olmo 3
• RLCSD: Reinforcement Learning with Contrastive On-Policy Self-Distillation

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks