SOURCE-LINKED INTELLIGENCE

Augustinian BabyLM: What Ostensive Definition Can and Cannot Teach a Small Language Model

arXiv · AI, language, vision and robotics · article · Sep 10, 2026 · UTC

A language model normally begins training with random word embeddings: whatever 'banana' means must be learned from training corpora. I implement St. Augustine's picture of word learning, meaning by ostension, for a small masked language model (DeBERTa) trained on 10M words: before training, visually grounded tokens receive embeddings derived from the image regions they label; other tokens start random. Visual initialization leaves a measurable imprint that lasts until the end of training. At the same time, the effect remains invisible under most BabyLM benchmarks, which probe abstract grammat

Read original source ↗ Open in workspace

recordType

paper

region

Global