SOURCE-LINKED INTELLIGENCE

KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU

arXiv · AI, language, vision and robotics · article · Sep 4, 2026 · UTC

Modern LLM agents operate in persistent workspaces whose accumulated history can exceed both GPU KV capacity and the model's native context window. Existing systems typically compact older context into summaries or retrieve it later as text, either losing fine-grained execution evidence or repeatedly prefilling content that the model has already processed. We present KVMem, a KV-context virtualization system that preserves overflowed workspace history as paged KV state across GPU memory, host memory, and NVMe. KVMem uses lightweight, model-native attention-space indexes to select relevant hist

Read original source ↗ Open in workspace

recordType

paper

region

Global