SOURCE-LINKED INTELLIGENCE

MemToC: Benchmarking Memory-Tool Conflict Resolution in Large Language Models

arXiv · AI, language, vision and robotics · article · Aug 26, 2026 · UTC

Tool-augmented LLMs must arbitrate between two fallible sources when a tool return conflicts with their parametric memory, yet existing evaluations measure source preference without establishing source correctness. We introduce MemToC, a controlled benchmark for post-tool-return arbitration with executable tools. MemToC comprises 6,504 evaluation episodes constructed from 542 quality-controlled factual questions, independently elicited model-specific closed-book answers, and controlled tool returns of known correctness. These components instantiate four source-correctness cases; tool-error and

Read original source ↗ Open in workspace

recordType

paper

region

Global