SOURCE-LINKED INTELLIGENCE

Temporal Self-Distillation: Faster Inference in Discrete Diffusion Language Models

arXiv · AI, language, vision and robotics · article · Sep 14, 2026 · UTC

Diffusion language models (dLLMs) promise fast inference by generating multiple tokens in parallel, but suffer severe performance degradation when parallel decoding is pushed too aggressively. We introduce Temporal Self-Distillation (TSD), a simple on-policy method that trains dLLMs for fast inference by distilling predictions across time. Specifically, TSD distills the model's denoising distribution at earlier timesteps toward its distribution at the final timestep at which a token is committed. This encourages earlier predictions to better anticipate the model's eventual output, enabling muc

Read original source ↗ Open in workspace

recordType

paper

region

Global