SOURCE-LINKED INTELLIGENCE

CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts

arXiv · AI, language, vision and robotics · article · Sep 9, 2026 · UTC

On-policy distillation (OPD) provides dense token-level supervision on student-generated trajectories. Although OPD performs strongly when teacher and student belong to the same model family, we find that its effectiveness degrades in cross-family settings even after tokenizer alignment, with substantially stronger external teachers offering little additional improvement. To understand this disconnect, we decompose the cross-family OPD signal into two components: an offset between a low-capability teacher-family reference and the student, and the within-family log-likelihood shift from that re

Read original source ↗ Open in workspace

recordType

paper

region

Global