SOURCE-LINKED INTELLIGENCE

Fidelity Is Not Enough: Dispatch-Level Instrumentation for Agentic Datasheet Extraction

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

One model passed our fidelity check without ever opening the datasheet. We found it while qualifying models for an internal extraction service: a structured-output constraint had silently disabled tool use, and the model answered anyway, with fabricated source text. Only the per-tool trace exposed it. Fidelity -- whether an extracted value matches the source -- is the standard measure for agentic document extraction, and it scores that run a success. We therefore log every tool call in an agentic benchmark of 25 hand-curated claims over three components, with 12 more on a fourth, 37 in all. Fr

Read original source ↗ Open in workspace

recordType

paper

region

Global