SOURCE-LINKED INTELLIGENCE

NCP-ArchPreview Technical Report: Moving towards Latent Space Language Models through Next Concept Prediction

arXiv · AI, language, vision and robotics · article · Sep 9, 2026 · UTC

We introduce NCP-ArchPreview, a latent-space language model that pushes autoregressive pretraining beyond standard next-token prediction (NTP). Alongside NTP, the model learns through Next Concept Prediction (NCP) to predict discrete concepts that span multiple tokens, introducing an explicit and more challenging concept-level objective while preserving standard token-level autoregressive generation. NCP-ArchPreview builds a latent space by constructing a product-quantized concept vocabulary directly from its hidden states, and subsequently learns to predict future concepts via a dedicated Con

Read original source ↗ Open in workspace

recordType

paper

region

Global