When Top-K Misses the Decision: Tool-Call Drift in Multi-Teacher On-Policy Distillation

arXiv:2607.07050 · cs.CL, cs.LG · Submitted 2026-07-08 · Read on arXiv

cs.CL, cs.LG

Submitted: 2026-07-08

Updated: 2026-09-05

Comments: 33 pages, 5 figures. Code and aggregate artifacts: https://github.com/shen-jiabin/decision-support-opd

Code: https://github.com/shen-jiabin/topk-support-opd

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Distillation — 44× in this paper · explained in 2 episodes
• Calibration — 22× in this paper · explained in 9 episodes
• On-Policy Distillation — 17× in this paper · explained in 3 episodes
• Accuracy — 16× in this paper · explained in 1 episode
• AUC — 13× in this paper · explained in 2 episodes
• Knowledge Distillation — 6× in this paper · explained in 15 episodes
• Benchmark — 3× in this paper · explained in 2 episodes
• Reinforcement Learning — 3× in this paper · explained in 4 episodes
• Trace — 3× in this paper · explained in 2 episodes
• Transformer — 3× in this paper · explained in 2 episodes
• Decision Risk — 2× in this paper · explained in 2 episodes
• Decode — 2× in this paper · explained in 2 episodes
• In International Conference on Machine Learning — 2× in this paper
• International Conference on Learning Representations — 2× in this paper
• Self-Distillation — 2× in this paper · explained in 7 episodes
• alpha — 2× in this paper
• Ablation — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper
• Core — 1× in this paper · explained in 2 episodes
• Decoupling — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Jensen–Shannon Divergence (JSD) — 1× in this paper · explained in 2 episodes
• Meeting of the Association for Computational Linguistics — 1× in this paper

Sources

• Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMs
• Counteraction-Aware Multi-Teacher On-Policy Distillation for General Capability Recovery with Domain Preservation
• Don't Ignore the Tail: Decoupling top-K Probabilities for Efficient Language Model Distillation
• On Group Relative Policy Optimization Collapse in Agent Search: The Lazy Likelihood-Displacement
• MiniLLM: On-Policy Distillation of Large Language Models
• Stabilizing On-Policy Distillation for MLLM Reasoning with Global Normalization
• Why Multi-Step Tool-Use Reinforcement Learning Collapses and How Supervisory Signals Fix It
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• Entropy-Aware On-Policy Distillation of Language Models
• ToDi: Token-wise Distillation via Fine-Grained Divergence Control
• DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs
• Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillation
• MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training
• Identified-Set Geometry of Distributional Model Extraction under Top-$K$ Censored API Access
• APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay
• A Survey of On-Policy Distillation for Large Language Models
• Reliability Gated Multi-Teacher Distillation for Low Resource Abstractive Summarization
• MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debate
• Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillation
• TIP: Token Importance in On-Policy Distillation

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering