SOURCE-LINKED INTELLIGENCE

RL Starts before RL: On Policy Distillation for Better Reinforcement Learning

arXiv · AI, language, vision and robotics · article · Sep 23, 2026 · UTC

Reinforcement learning (RL) improves reasoning, but its performance depends on the policy from which training begins. We study on-policy distillation (OPD) as a preparation stage for RL and ask whether its benefits extend beyond improvements in the distilled model's initial accuracy. Under shared RL settings, students initialized with OPD reach higher final performance than those trained with direct RL or supervised fine-tuning followed by RL. This advantage can emerge even when OPD produces little immediate improvement in accuracy. Pre-RL Pass@k does not fully explain the benefit: similar or

Read original source ↗ Open in workspace

recordType

paper

region

Global