SOURCE-LINKED INTELLIGENCE

One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation

arXiv · AI, language, vision and robotics · article · Aug 26, 2026 · UTC

On-policy distillation trains a language model on its own generations while a teacher scores them token by token. It combines the dense supervision of imitation learning with the on-policy sampling of reinforcement learning. But it requires a second, larger model to act as teacher. On-Policy Self-Distillation (OPSD) removes that cost. The teacher is the model itself, conditioned on privileged information the student will not have at test time, such as a reference solution, a plan, or environment feedback. The teacher is no stronger than the student, only better informed. Early results were pro

Read original source ↗ Open in workspace

recordType

paper

region

Global