SOURCE-LINKED INTELLIGENCE

A Formal Limitation on Learning Human Language From Textual Corpora

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

Can a listener recover what a speaker means from the form of an utterance alone? We answer this question information-theoretically, and for a listener given by any featurizer of text, including the hidden states of contemporary large language models. Modeling language use as a joint distribution over meanings, contexts, and utterances, we derive upper bounds on the probability that a decoder recovers a speaker's intended meaning from a representation of the utterance. The bounds are governed by the uncertainty that form leaves about meaning, which splits into an irreducible part and a part tha

Read original source ↗ Open in workspace

recordType

paper

region

Global