SOURCE-LINKED INTELLIGENCE

Exact Finite Attention Responses From RoPE Derivatives

arXiv · AI, language, vision and robotics · article · Sep 12, 2026 · UTC

We derive exact local responses for attention interventions, allowing candidate edits to be scored from a cached baseline and one backward pass. The starting point is the RoPE derivative $\partial_p z(p) = A z(p)$: its integral gives the finite positional displacement, which we carry through the softmax without linearising either rotation or normalisation. The resulting predictions achieve 95.36--96.52% sign accuracy across 92,160 executed positional edits on 768 held-out prompt sets, reducing answer-margin MAE by 73.6--82.5% against the positional Jacobian and by 36.2--50.9% against zero. For

Read original source ↗ Open in workspace

recordType

paper

region

Global