SOURCE-LINKED INTELLIGENCE

AutoTuneBench: Trustworthy Measurement for Agent Auto-Tuning of LLM Serving Engines

arXiv · AI, language, vision and robotics · article · Sep 16, 2026 · UTC

Large language model agents tune GPU kernels and serving engines through a closed loop of propose, measure, and keep, but the measurements behind this loop are not trustworthy. We characterize four failure modes from a four-day pilot corpus of 619 model calls: strawman baselines manufacture speedups, absolute times do not transfer across machines, saturated tasks nullify comparisons, and infrastructure defects impersonate science. We present AutoTuneBench, a benchmark and measurement protocol that makes trust architectural. The protocol is frozen as code with test-enforced provenance; a databa

Read original source ↗ Open in workspace

recordType

paper

region

Global