SOURCE-LINKED INTELLIGENCE

Off-Policy Evaluation for Semantic ID Recommenders: Does the Model's Own Code Hierarchy Help?

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

Generative recommenders increasingly emit semantic IDs (SIDs): each item is a short sequence of hierarchical discrete codes from a residual quantizer, decoded autoregressively. Before spending scarce A/B-test, a team may decide offline which decoder or reranking variants are worth testing - a job for off-policy evaluation (OPE). We ask a simple question: can the model's own SID tree serve as the action abstraction for that OPE? Our answer has three parts. (i) Under the near-argmax logging real recommenders use, per-item OPE is hopeless - as item-level effective sample size is usually small on

Read original source ↗ Open in workspace

recordType

paper

region

Global