Online Learning-to-Defer with Varying Experts
stat.ML, cs.LG
Submitted: 2026-05-12
Updated: 2026-08-28
License: http://creativecommons.org/licenses/by/4.0/
The gist: Learning-to-Defer (L2D) methods route each query either to a predictive model or to external experts.
Terminology
Abstract
Learning-to-Defer (L2D) methods route each query either to a predictive model or to external experts. Real-world deployments require handling streaming data, changing expert availability, shifting expert reliability, and feedback observed only for the selected action. We introduce an online multiclass L2D algorithm that combines queried-action bandit feedback with a dynamically varying pool of experts. Let N=n+n e, let B bound the Frobenius norm of the linear score matrix, and let ρ bound the augmented input norm. Assuming linear calibration and zero surrogate minimizability gap for the projected comparator class, our method achieves expected true-deferral regret O((BN 3/2ρ+1)T 2/3), improving to O(BN 3/2ρ sqrt T+B 2N 3ρ 2) under a concentrated-score condition. The analysis combines an online H-consistency transfer bound with projected online convex optimization. Experiments on synthetic and real-world datasets demonstrate selective routing under varying expert availability and reliability.
Sources
- Budgeted Multiple-Expert Deferral
- Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
- Principled Algorithms for Optimizing Generalized Metrics in Multi-Label Learning
- Generalized Distributional Alignment Games for Unbiased Answer-Level Fine-Tuning
- Beyond Augmented-Action Surrogates for Multi-Expert Learning-to-Defer
- Consistent Learning-to-Defer with Expert-Conditional Advice
- Learning-to-Defer in Non-Stationary Time Series via Switching State-Space Models
- Online Learning: A Modern Introduction Using Convex Optimization
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- Scalable Posterior Uncertainty for Flexible Density-Based Clustering
- Efficient quadratic entropy with distance sketches
- Embedding-Bias in Conditional Independence Testing