SOURCE-LINKED INTELLIGENCE

ORQA: An Occupation-Realistic Question and Answer Framework for LLM Professional Knowledge

arXiv · AI, language, vision and robotics · article · Sep 11, 2026 · UTC

We present ORQA, a method for testing occupation-level knowledge in large language models. Prior methods either map abstract LLM skills to occupations via task definitions or utilize expert knowledge which is difficult to obtain at scale and expensive. ORQA complements both of these methods by connecting O*NET occupations to trusted occupation-specific websites (such as regulatory agencies, licensing bodies, professional organizations, and government publications) and converting these into source-traceable question-answer pairs. A combination of an automated pipeline and human review produces

Read original source ↗ Open in workspace

recordType

paper

region

Global