SOURCE-LINKED INTELLIGENCE

Tasks over Application Manuals: Revealing Gaps in Long-Horizon Procedural Reasoning for Language Models

arXiv · AI, language, vision and robotics · article · Sep 11, 2026 · UTC

Large language models (LLMs) have achieved strong performance on a wide range of natural language tasks, and recent benchmarks suggest that they are increasingly adept at multi-hop reasoning. However, these benchmarks are typically short-horizon, requiring only a small number of retrieval or inference steps, and provide limited evidence of reliability on real-world tasks that involve following manuals spanning hundreds of pages with complex, interdependent guidelines. In this paper, we introduce Tasks over Application Manuals (TAM), a benchmark for evaluating long-horizon procedural reasoning.

Read original source ↗ Open in workspace

recordType

paper

region

Global