SOURCE-LINKED INTELLIGENCE
Why Pretraining Fails to Share Cross-Lingual Knowledge
Large Language Models (LLMs) have made remarkable progress in the processing and modeling of many languages. Yet, unlike human multilinguals, they exhibit surprisingly limited cross-lingual knowledge transfer. While this limitation is well documented, its origins during multilingual training remain unclear. We pretrain 360M- and 7B-parameter LLMs and show that poor cross-lingual knowledge generalization emerges during pretraining and persists under standard interventions. To isolate its cause, we employ a controlled bilingual pretraining setting using two copies of the same language, sharing i
Read original source ↗ Open in workspace
- recordType
- paper
- region
- Global
Evidence & attribution
- arXiv · Artificial Intelligence · 2026-09-16T18:02:29.000Z
- arXiv · AI, language, vision and robotics · 2026-09-16T18:02:29.000Z
First collected: 2026-09-19T20:26:32.566Z. This is not the publication date.