SOURCE-LINKED INTELLIGENCE

Benchmark Contamination: A Taxonomy Organized by Defeated Mitigation

arXiv · AI, language, vision and robotics · article · Aug 29, 2026 · UTC

A benchmark score is a joint property of the model, the evaluation harness, the elicitation budget, the sampled population, and contamination status. Leaderboards publish the model and the score, so capability and leakage stay observationally equivalent. Existing taxonomies classify contamination for automated detection, not the question a reporter faces at publication: given the mitigations already applied, which validity threats remain open? We introduce a taxonomy organized by the mitigation each type defeats -- direct, derivative, temporal, distributional, and acquired -- spanning training

Read original source ↗ Open in workspace

recordType

paper

region

Global