SOURCE-LINKED INTELLIGENCE

Tail-Likelihood Reinforcement Learning

arXiv · AI, language, vision and robotics · article · Sep 2, 2026 · UTC

Reinforcement learning typically optimizes average reward. For generative policies, the average can hide an important distinction: two policies can achieve the same mean reward while having very different chances of producing a rare but high-reward rollout. This matters as sampling increases during training and inference, since its benefit depends on retaining probability mass on high-reward outcomes. We propose to optimize this coverage directly. Rather than considering only expected reward, we consider all of its upper tails: for each reward threshold, how likely is the policy to exceed it?

Read original source ↗ Open in workspace

recordType

paper

region

Global