SOURCE-LINKED INTELLIGENCE

Sequential Beats Joint: On the Interplay between On-Policy Distillation and RLVR

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

Reinforcement learning with verifiable rewards (RLVR) and on-policy distillation (OPD) have emerged as two dominant methods for post-training reasoning LLMs. Prior work uses OPD's dense token-level supervision to complement the sparse RL reward, fusing the two signals within a single step: either as a \emph{weighted-additive combination} or a \emph{teacher-modulated rescaling} of the RL advantage. In this paper, we show that a simple two-stage scheme, OPD-then-RL, consistently outperforms pure OPD, pure RLVR, and all such joint baselines across logic and math reasoning benchmarks. Beyond the e

Read original source ↗ Open in workspace

recordType

paper

region

Global