SOURCE-LINKED INTELLIGENCE

The Audit Decides the Verdict: Instrument Effects Rival Demographic Bias in LLM Decision Audits

arXiv · AI, language, vision and robotics · article · Sep 8, 2026 · UTC

Whether a language model looks demographically biased can depend on how the audit asks its question. A charitable-aid benchmark reports that the same models favor minority applicants when rating requests one at a time and penalize some when ranking side by side. We test whether that reversal generalizes to hiring, lending, and medical triage: 40,726 requests to five models, applications differing only in the applicant's name, and a primary test fixed before collection. It does not. None of 36 planned contrasts survives correction. The rating advantage keeps its sign at roughly half the publish

Read original source ↗ Open in workspace

recordType

paper

region

Global