SOURCE-LINKED INTELLIGENCE

K-Bench: a clinically calibrated benchmark for evaluating large language models in high-risk mental health conversations

arXiv · AI, language, vision and robotics · article · Sep 14, 2026 · UTC

People increasingly use large language models (LLMs) for mental health support, yet their safety in evolving, high-risk conversations remains poorly characterised. We developed K-Bench, a clinician-calibrated, protected benchmark evaluating 125 model configurations representing 33 base models from 14 providers across a fixed cohort of 200 multi-turn vignettes involving suicide, self-harm, domestic violence, substance misuse, and no-risk presentations. Synthetic patient conversations showed substantial distributional overlap with real human-AI conversations. A frozen GPT-4o judge achieved 94.2%

Read original source ↗ Open in workspace

recordType

paper

region

Global