SOURCE-LINKED INTELLIGENCE

How Correct Is Your Answer? A Semantic Correctness Framework for Open QA Evaluation

arXiv · AI, language, vision and robotics · article · Sep 1, 2026 · UTC

Reliable evaluation of open-ended question answering remains a bottleneck for measuring answer correctness of modern LLMs. Unlike multiple-choice tasks, free-form answers may be correct in many surface forms and may fail in qualitatively different ways, including incompleteness, contradiction, overgeneration, and endorsement of false premises. Existing judgment-based and similarity-based metrics often collapse these distinctions. We address this gap with three reusable contributions. First, we introduce a semantic correctness taxonomy that assigns open-ended answers to eight ordered classes, s

Read original source ↗ Open in workspace

recordType

paper

region

Global