SOURCE-LINKED INTELLIGENCE

Rethinking On-Policy Distillation of Large Language Models II: One Training Example

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

On-policy distillation (OPD) combines student-generated rollouts with dense token-level supervision from a teacher. Existing work has mainly studied its algorithmic behavior, leaving the role of training data unclear. We examine this role at the data-minimal limit by training on a single query. One-shot OPD keeps improving for hundreds of steps and recovers most of full-data OPD's gain across task domains and model families. We explain this result through the states visited during training and the rate at which the student aligns with the teacher. We measure \emph{state coverage}, the fraction

Read original source ↗ Open in workspace

recordType

paper

region

Global