SOURCE-LINKED INTELLIGENCE

CALICO: A Human-Centered, Codebook-Aligned System for Annotation

arXiv · AI, language, vision and robotics · article · Sep 13, 2026 · UTC

Large language models are increasingly used to scale codebook-based annotation in scientific research, but existing workflows provide limited support for translating domain experts' codebooks into reliable, revisable, and auditable prompts. Prompts are often treated as fixed instructions and hidden from annotators, making it difficult for non-technical domain experts to diagnose and correct model behavior when outputs violate codebook guidelines. In this paper, we present CALICO, a human-centered, codebook-aligned annotation workflow that treats prompts as editable, versioned, and optimizable

Read original source ↗ Open in workspace

recordType

paper

region

Global