SOURCE-LINKED INTELLIGENCE

When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token Analysis

arXiv · AI, language, vision and robotics · article · Sep 14, 2026 · UTC

Large language model (LLM) agents allocate test-time compute adaptively as they revise solutions, use tools, explore alternatives, and decide when to stop. This test-time strategy makes it difficult to measure how agent performance scales. We study open-ended tasks that provide continuous scores for intermediate submissions, making progress observable throughout long trajectories. We propose Elo-per-token analysis, which tracks the best solution found at each token budget and uses a Bradley-Terry model to aggregate within-task orderings into Elo ratings across tasks with different score scales

Read original source ↗ Open in workspace

recordType

paper

region

Global