SOURCE-LINKED INTELLIGENCE

Prediction-Powered Smoothing and Validation for Disaggregated AI Evaluation

arXiv · Artificial Intelligence · article · Sep 17, 2026 · UTC

Evaluating an AI system requires disaggregated assessment, as performance varies across domains such as benchmark task types or conversation types in deployed agents. Exhaustive testing is expensive, so evaluation rests on a sample of labeled units. We treat the evaluation set as a finite population and seek accurate point and interval estimates of each domain mean. Direct estimators, including prediction-powered inference (PPI), use only a domain's own labels and are imprecise where labels are few. Small area estimation addresses this problem, and we build on it to develop an integrated workf

Read original source ↗ Open in workspace

recordType: paper
region: Global

Evidence & attribution

arXiv · AI, language, vision and robotics · 2026-09-17T17:42:29.000Z
arXiv · Artificial Intelligence · 2026-09-17T17:42:29.000Z

First collected: 2026-09-19T20:26:32.566Z. This is not the publication date.