SOURCE-LINKED INTELLIGENCE

Generalization through Lexical Abstraction in Transformer Models: The Case of Functional Words

arXiv · AI, language, vision and robotics · article · Sep 17, 2026 · UTC

Pronouns, adverbs and other functional words (such as they, her, somewhere, there) are often used in language to replace concrete nouns or phrases, when their properties - such as gender, grammatical number - provide sufficient information for the given context. Do pretrained transformer models encode such functional words in a manner that allows them to be used like humans do? Can language models recognize the syntactic and semantic parallelism of sentences such as "The researchers wrote the paper" and "They wrote it", which relies on such lexical abstraction? We map these linguistic question

Read original source ↗ Open in workspace

recordType

paper

region

Global