SOURCE-LINKED INTELLIGENCE

Amortizing Scaling Law Construction Costs

arXiv · AI, language, vision and robotics · article · Sep 4, 2026 · UTC

Scaling laws guide the design choices for training large foundation models, but deriving them involves training an exhaustive grid over hyperparameters, token budgets, and parameter counts, which is computationally expensive. Fitting a scaling law, however, only requires the best-loss frontier across compute scales, discarding most of the trained configurations. We propose a framework for efficient scaling law construction that formulates data collection as a Bayesian optimization problem, and introduce metrics for comparing scaling law fitting methods under constrained compute budgets. We fin

Read original source ↗ Open in workspace

recordType

paper

region

Global