SOURCE-LINKED INTELLIGENCE

The Misery of Mechanistic Interpretability: A Formal Perspective

arXiv · AI, language, vision and robotics · article · Sep 14, 2026 · UTC

Mechanistic interpretability has become the dominant lens for understanding frontier language models, as their inner workings are complex and inherently black boxes. To gain insights into these models, interpretable replacement networks (IRNs) are trained at all layers, exposing interpretable features through sparsely activated neurons. However, the faithfulness of an IRN is usually evaluated only empirically on clean data, and we show that even semantically minor input perturbations flip the dominant IRN features-and thus the human-understandable interpretation-across five open-weight model f

Read original source ↗ Open in workspace

recordType

paper

region

Global