SOURCE-LINKED INTELLIGENCE

Distillation as Probability Transport: Routed On-Policy Distillation

arXiv · AI, language, vision and robotics · article · Sep 8, 2026 · UTC

On-policy distillation (OPD) transfers teacher knowledge on student-generated trajectories, but efficient sampled objectives reduce the teacher distribution to scalar credit on individual tokens. Such credit indicates whether a token should gain or lose probability, yet leaves the corresponding redistribution unspecified. We recast OPD as teacher-guided probability transport and propose RouteOPD (Routed On-Policy Distillation), which decomposes local teacher--student disagreement into student-excess sources and teacher-deficit destinations and couples them into explicit transport pairs. RouteO

Read original source ↗ Open in workspace

recordType

paper

region

Global