SOURCE-LINKED INTELLIGENCE

When Rubrics Fail: Hallucinations Reveal Blind Spots in Medical AI Evaluation

arXiv · AI, language, vision and robotics · article · Sep 11, 2026 · UTC

Hallucinations can undermine clinician trust in LLMs, making it important that evaluation methods capture clinically relevant errors. Rubric-based evaluation has become the leading approach for assessing LLMs in medicine, but it is unclear whether rubric scores reflect such errors. We first study this in a controlled setting using MedHallu, finding that more specific rubrics better distinguish correct from hallucinated responses. To test this systematically, we develop a taxonomy of medical hallucination types and a clinician-validated error-injection pipeline that creates matched correct and

Read original source ↗ Open in workspace

recordType

paper

region

Global