SOURCE-LINKED INTELLIGENCE

ContractEval: Query-Conditioned Execution Matching for Procedural Instruction Conformance

arXiv · AI, language, vision and robotics · article · Sep 8, 2026 · UTC

As LLM agents move from answering questions to carrying out procedures, failures can be unwarranted rather than visibly wrong: the final response looks acceptable even though the system skipped the check, branch, dependency, or invariant that made the answer justified. Output-only evaluation sees the answer, and trace-aware judging sees activity, but neither identifies which obligations were active for the query. We introduce CONTRACTEVAL, a diagnostic framework for making those active obligations explicit. It represents procedural instructions as query-active obligations and matches them agai

Read original source ↗ Open in workspace

recordType

paper

region

Global