SOURCE-LINKED INTELLIGENCE

On-Policy Self-Distillation in Diffusion Models

arXiv · AI, language, vision and robotics · article · Aug 25, 2026 · UTC

Reinforcement learning can align diffusion models with human preferences and task-specific objectives, but endpoint rewards do not specify how an intermediate denoising prediction should change. We introduce DiffusionOPSD as an on-policy self-distillation framework that converts image-level reward guidance into explicit targets for clean-output predictions at sampled queries. At each outer iteration, a frozen behavior policy generates trajectories and supplies query states and anchors. Reward gradients construct bounded positive and negative targets around each anchor. The trainable policy fit

Read original source ↗ Open in workspace

recordType

paper

region

Global