SOURCE-LINKED INTELLIGENCE

Intra-Prompt Parallel Decoding for Common-Context Question Answering

arXiv · AI, language, vision and robotics · article · Sep 4, 2026 · UTC

In common-context question answering (CCQA) tasks, multiple input questions share a common context to base their answers from. However, Large Language Models typically generate each answer using an independent prompt. While existing batching and caching techniques help improve parallelism and reduce repeated computations, the separation of questions across prompts limits the achievable speedup, as modern GPUs are underutilized due to a memory bottleneck during attention. We present Intra-Prompt Parallel Decoding (IPPD), a novel inference method that answers multiple common-context questions in

Read original source ↗ Open in workspace

recordType

paper

region

Global