SOURCE-LINKED INTELLIGENCE

Learning with Synthetic Data via SGD in High-Dimensional Linear Regression

arXiv · AI, language, vision and robotics · article · Sep 9, 2026 · UTC

Synthetic data has become a promising way to scale model training beyond limited human-generated data but it may also induce strong model collapse (Dohmatob et al., 2024), where any fixed fraction of synthetic data prevents model performance from improving under data scaling, leaving a non-vanishing excess risk floor. In this paper, we study how synthetic data affects the generalization of one-pass SGD in high-dimensional linear regression with model shift. We establish finite-sample risk bounds for mixed and two-stage training, separating standard bias and variance from source-mismatch effect

Read original source ↗ Open in workspace

recordType

paper

region

Global