SOURCE-LINKED INTELLIGENCE

Characterization of Request and Token Energy Costs for LLM Inference Workloads on GPU Platforms

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

Large language model (LLM) inference serving is priced by tokens, but GPU energy is consumed over inference windows. This accounting mismatch makes token-normalized metrics incomplete, since average output-token energy can decrease even when total request energy increases. We characterize this behavior with a decomposed energy model: a fixed one-time prefill with a fixed generation setup cost, while each output-token generation step adds marginal step energy. We evaluate this LLM inference energy model on NVIDIA H100 and H200 GPUs across dense and mixture-of-experts (MoE) models, reporting bot

Read original source ↗ Open in workspace

recordType

paper

region

Global