SOURCE-LINKED INTELLIGENCE

ReTrace: Rejected-Trajectory Conditioning for Speculative Decoding

arXiv · AI, language, vision and robotics · article · Aug 30, 2026 · UTC

Speculative decoding accelerates autoregressive language model inference by having a lightweight draft model propose multiple candidate tokens, which are then verified in parallel by a larger target model. However, after the first rejection, standard prefix-based verification discards the remaining draft suffix, so the computation spent generating and verifying those positions does not contribute to decoding progress. Focusing on DFlash, we show that rejected positions in a rejected suffix may still align with the target continuation, indicating that the draft model can retain useful semantic

Read original source ↗ Open in workspace

recordType

paper

region

Global