SOURCE-LINKED INTELLIGENCE

Through the Looking Glass: Directly Reading and Writing Transformers

arXiv · AI, language, vision and robotics · article · Sep 9, 2026 · UTC

How many of a transformer's components decide a token? Counted by the absolute value of each unit's and channel's contribution to the logit, one prediction rests on thousands to hundreds of thousands of them. But contributions are signed, and across eighteen models the mass pushing away from the predicted token is a median of seven times the mass carrying it. Divide by the net and the count is dozens: on the baseline, 53 components carry ninety percent of a prediction, 13 it cannot survive losing, and 8 suffice to produce it alone. Across twelve models trained elsewhere, 124M to 7B parameters,

Read original source ↗ Open in workspace

recordType

paper

region

Global