SOURCE-LINKED INTELLIGENCE

MeshKV: A Network-on-Chip KV Cache Fabric for Scalable Transformer Decoding Accelerators

arXiv · AI, language, vision and robotics · article · Sep 16, 2026 · UTC

Autoregressive transformer decoding is constrained by irregular key-value (KV) cache movement on tiled accelerators. Prior compression and DRAM-placement systems still concentrate traffic on centralized memory paths that bottleneck long-context serving. We present MeshKV, a KV cache fabric that moves blocks as packetized flows over a lightweight NoC. It co-designs (i) TaKV affine striping to spread homes and cut hotspot load, (ii) Mare multicast with verified duplicate suppression, and (iii) Pad, which overlaps prefetch, tile multiply, and streaming softmax behind credit-aligned FIFOs. Togethe

Read original source ↗ Open in workspace

recordType

paper

region

Global