SOURCE-LINKED INTELLIGENCE

A Repeated-Measurement Study for Cultural Analytics of English Song Lyrics Using Five Large Language Models

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

Large language models (LLMs) are increasingly used to annotate cultural texts at scales that are impractical for human coders. However, before their outputs are treated as measurements of latent social constructs, it is necessary to establish whether those measurements are reliable. This study evaluates five LLMs as zero-shot annotators of four social constructs expressed in English song lyrics: self-esteem, self-control, seeking belonging, and seeking recognition. Using repeated annotations of a large lyric corpus, we examine three properties of LLM-based measurement: consistency across repea

Read original source ↗ Open in workspace

recordType

paper

region

Global