SOURCE-LINKED INTELLIGENCE

Limitations of Automated Simulatability: LLM Simulators Can Bypass Explanations

arXiv · AI, language, vision and robotics · article · Sep 8, 2026 · UTC

Simulatability is an evaluation protocol for explanations that quantifies their usefulness by how well they help a user predict a task model's outputs. Since human evaluation is costly, automated simulatability replaces human explainees with LLM simulators, as proposed in ConSim (Poché et al., 2025) for large-scale experiments. We qualitatively replicate and extend ConSim's ranking of explanation methods across the tested datasets, explanation families, and simulator LLMs, and identify two limitations. First, when class names are meaningful, simulators can obtain high simulatability by solving

Read original source ↗ Open in workspace

recordType

paper

region

Global