SOURCE-LINKED INTELLIGENCE

DualOPSD: Adaptive Privileged Teachers for On-Policy Self-Distillation

arXiv · AI, language, vision and robotics · article · Aug 26, 2026 · UTC

On-policy self-distillation (OPSD) uses a privileged copy of the student model to provide dense supervision without an external teacher. OPSD keeps this privileged teacher fixed, even though the student distribution and output style change during training. We propose DualOPSD, an asymmetric alternating framework that adapts both policies. The student first learns from the privileged teacher. The teacher then moves toward the updated student distribution on the same student trajectory. This update makes later supervision responsive to the learner and does not require another rollout. On Qwen3-8

Read original source ↗ Open in workspace

recordType

paper

region

Global