SOURCE-LINKED INTELLIGENCE

A Smaller Transformer in Your Transformer

arXiv · AI, language, vision and robotics · article · Sep 17, 2026 · UTC

Recent findings indicate that Vision Transformers settle into locally similar computational phases, implying a level of depthwise computational redundancy. However, existing methods to exploit this redundancy either fail to reduce inference compute or severely degrade model expressivity. In this work, we formalise a unified view of block redundancy that decouples the geometry from specific surrogate interventions. We then introduce Transformer-Within-Transformer (TWT), a post-hoc method that fuses contiguous groups of redundant layers into a single learned surrogate layer. TWT reduces paramete

Read original source ↗ Open in workspace

recordType

paper

region

Global