SOURCE-LINKED INTELLIGENCE

Temporal Recurrence Favors Fewer Layers

arXiv · AI, language, vision and robotics · article · Sep 11, 2026 · UTC

In streaming tasks, recurrent models can carry latent computation across time, allowing each update to build on representations produced earlier. This raises a basic question: once temporal recurrence provides sequential computation across steps, how much depth is still needed within each step? Prior work has shown that recurrence can make shallow models competitive. We instead study this question as a compute-allocation problem, varying within-step depth, expert width, and the number of parallel experts per layer across several compute budgets. For each budget, we compare the best observed re

Read original source ↗ Open in workspace

recordType

paper

region

Global