SOURCE-LINKED INTELLIGENCE

Data-Efficient Language Modeling: From Frontier Advancement to Principle-Guided Model Improvement

arXiv · AI, language, vision and robotics · article · Sep 9, 2026 · UTC

Learning from limited text requires models to use context, generalize to new inputs, and retain useful capabilities. Qiushi Engine conducted a long-horizon, end-to-end autonomous research program on BabyLM 2026 Strict-Small, within 10 million corpus words and 100 million cumulative word presentations. Three stages connected frontier advancement, principle discovery, and principle-guided model improvement. Stage I combined compact restatements, budget reinvestment, and residual incremental learning to build a frontier model. Stage II found that exact repetition and aligned restatement produce d

Read original source ↗ Open in workspace

recordType

paper

region

Global