SOURCE-LINKED INTELLIGENCE

To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals

arXiv · AI, language, vision and robotics · article · Sep 17, 2026 · UTC

Speculative Decoding (SD) has significantly accelerated Large Language Model (LLM) inference, yet existing approaches face a fundamental tradeoff between two drafting strategies: neural drafting and context-based copying. Neural drafts (e.g., EAGLE3) provide robust performance across diverse text settings, while copy-based methods achieve higher speedups in copy-intensive regimes by generating candidates faster and exploiting long repetition spans for near-perfect speculation. We analyze existing copy-based methods and find that they are prone to accidental repetitions where surface-level n-gr

Read original source ↗ Open in workspace

recordType

paper

region

Global