paper
arXiv cs.CL
November 9th, 2026 at 5:00 AM

Safety and accuracy diverge across deployment conditions in clinical large language models

arXiv:2605.04039v2 Announce Type: replace Abstract: Clinical large language models (LLMs) are commonly scaled by increasing model size, context length, retrieval complexity, or inference-time compute, on the assumption that higher accuracy implies safer behavior. In medicine, where a few confident or evidence-contradicting errors can outweigh average performance, this assumption may be incomplete. We introduce Safety-Focused Evaluation of Scaling (SaFE-Scale) and instantiate it on RadSaFE-200, a 200-question text-based radiology multiple-choice benchmark with clinician-defined clean evidence, conflict evidence, and option-level labels for high-risk error, unsafe answer, and evidence contradiction. Thirty-four local LLMs were evaluated across six deployment conditions spanning closed-book, evidence-grounded, standard and agentic retrieval- augmented generation (RAG), and max-context prompting. Clean clinician-written evidence, used as an upper-bound, produced the strongest joint gain, raising mean accuracy from 73.5% to 94.1% and reducing high-risk error from 12.0% to 2.6% and dangerous overconfidence from 8.0% to 1.6%. Standard and agentic RAG, max-context prompting, and inference-time compute improved selected metrics but did not reproduce this safety profile. Clinically, consequential errors concentrated in a small subset of questions, and variance decomposition placed deployment condition above model family on every safety metric. Within this evaluation, clinical LLM safety did not track accuracy scaling but emerged as a deployment-level property.

#ai
#llm

Get stories like this every Monday

The week's most important AI news in one email. Free, unsubscribe anytime.

Score: 2.88

Engagement proxy: 0

Canonical link: https://arxiv.org/abs/2605.04039