SOURCE-LINKED INTELLIGENCE

The Gradient Does Not See Rank: Rank-Indifference in Matrix-CODI on ProsQA

arXiv · AI, language, vision and robotics · article · Sep 2, 2026 · UTC

Continuous chain-of-thought models compress reasoning into latent tokens. Matrix-valued variants, which route each latent token through a d x d matrix bottleneck, introduce rank as a single-sample structural observable on the latent matrix Z. If matrix latents carry parallel reasoning paths via superposition, rank should track them, and truncating Z to low rank should hurt accuracy on tasks whose solutions plausibly require multiple components. Across four training regimes of a matrix-CODI model (three on ProsQA, one on GSM8K-Aug below the learning threshold), the rank-k projection ablation cu

Read original source ↗ Open in workspace

recordType

paper

region

Global