SOURCE-LINKED INTELLIGENCE

Molecular Déjà Vu: Digit-Level Retrieval of Published Values in Frontier Language Models

arXiv · AI, language, vision and robotics · article · Sep 4, 2026 · UTC

Large language models (LLMs) are increasingly evaluated on molecular property benchmarks, but accuracy cannot distinguish a model that predicts a property from one that retrieves a published number. We audit 22 frontier models on 12 regression benchmarks for verbatim retrieval and find that it is widespread but relatively benchmark-specific: on five datasets more than $50\%$ of the LLMs show verbatim retrieval, while on the remaining datasets it appears only in isolated cells. We run our experiments at two reasoning levels and find that reasoning changes retrieval. The same experiments, on the

Read original source ↗ Open in workspace

recordType

paper

region

Global