SOURCE-LINKED INTELLIGENCE

AgentKV: Phase-Aware KV Eviction for Agentic LLMs

arXiv · AI, language, vision and robotics · article · Sep 14, 2026 · UTC

Agentic serving can consume orders of magnitude more tokens than chatbot workloads, stressing both KV-cache capacity and decode-time bandwidth. Most KV-eviction methods score cached keys against representative queries drawn from the most recent tokens, assuming future attention resembles recent attention. We show that agentic generation violates this assumption: future queries form a mixture over think, act, tool, and others phases, and principal-angle analysis shows these components occupy measurably different query subspaces, so recency representatives systematically undervalue keys that upc

Read original source ↗ Open in workspace

recordType

paper

region

Global