SOURCE-LINKED INTELLIGENCE

Detectable Only Where It Is Confounded: What Verified Duplication Counts Say About Membership Evidence in Language Models

arXiv · AI, language, vision and robotics · article · Sep 9, 2026 · UTC

When a language model finds a sentence unusually cheap to predict, it is tempting to conclude that the sentence was in its training data. Almost every published test of that inference has had to guess which sentences were in the training data, the members, and which were not. This paper removes the guessing. Two model families, OLMo-2 and Pythia, publish their pretraining corpora, and a public index over those corpora returns the exact number of times any sentence appeared in each. Those counts make three questions answerable directly. The answers form a pincer, closing from two sides. At the

Read original source ↗ Open in workspace

recordType

paper

region

Global