SOURCE-LINKED INTELLIGENCE

Prefix Sharing Is a Sorting Problem

arXiv · AI, language, vision and robotics · article · Sep 12, 2026 · UTC

LLM serving reuses KV cache by exact prefix match, so when a prompt is assembled from a set of reusable pieces -- retrieved passages, tool definitions, few-shot exemplars -- the order chosen for those pieces determines how much computation can be shared. Every deployed system fixes that order by a single global convention. We prove this is optimal only when requests contain at most two pieces, and asymptotically wrong in general. Our main result is a structure theorem: the minimum prefix-trie cost equals min_H sum_x w(x) t_x(H) over binary hierarchies H on the requests, where t_x(H) is the can

Read original source ↗ Open in workspace

recordType

paper

region

Global