SOURCE-LINKED INTELLIGENCE

Minibatch persistency, eight years later: what batch reuse costs in steps and joules, and what it saves in data

arXiv · AI, language, vision and robotics · article · Sep 12, 2026 · UTC

Minibatch persistency reuses data instead of reading it: rather than drawing a fresh minibatch at every optimizer step, it takes K consecutive steps on the same one. Absorbed into data echoing in 2019, it has carried one objection -- that reuse merely imitates a larger learning rate -- and no baseline tuned as carefully as the method itself. This paper runs the missing test. A pre-registered study trains a 49M-parameter Transformer on FineWeb-Edu at minibatch size B in {32, 128, 512}, 8 seeds per cell, tuning the learning rate separately for every batch size and every arm, against a reuse-free

Read original source ↗ Open in workspace

recordType

paper

region

Global