SOURCE-LINKED INTELLIGENCE

REER-PT: Reverse-Engineered Reasoning for Perplexity-Guided Pre-training Data Augmentation

arXiv · AI, language, vision and robotics · article · Aug 31, 2026 · UTC

As language-model compute continues to scale, high-quality training data is becoming an increasingly important bottleneck. Conventional next-token prediction supervises what follows a context but leaves the intermediate reasoning behind that continuation implicit. We introduce \textbf{REER-PT}, a scalable framework that extends Reverse-Engineered Reasoning (REER) to raw pre-training data. REER-PT identifies continuations that are difficult to predict but can still be inferred from the preceding context, and inserts concise reasoning annotations that reconstruct the missing connection between c

Read original source ↗ Open in workspace

recordType

paper

region

Global