SOURCE-LINKED INTELLIGENCE

SHELF: A Synthetic Harness for Multi-Task Bibliographic Benchmarking

arXiv · AI, language, vision and robotics · article · Sep 2, 2026 · UTC

Libraries and archives manage large collections with limited staff and computing budgets, yet common benchmarks do not systematically test their bibliographic work. They need to know which methods work for their tasks and what those methods require to run. SHELF, the Synthetic Harness for Evaluating LLM Fitness, addresses this gap. It is a Python system that turns labelled taxonomies, writing specifications, and a generation budget into controlled benchmark data and evaluation tasks. This first release contains 62,899 model-written documents based on Library of Congress vocabularies, with task

Read original source ↗ Open in workspace

recordType

paper

region

Global