SOURCE-LINKED INTELLIGENCE

Multi-step Proximal Policy Improvement in Offline Reinforcement Learning

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

Offline reinforcement learning (RL) must reconcile two competing requirements: policy updates should stay near dataset-supported actions to keep value estimates reliable, yet meaningful gains often require moving beyond the behavior distribution. We develop a geometric view of offline actor updates by modeling policies as a probability manifold endowed with a chosen metric geometry. Under this lens, a broad class of offline actor objectives can be interpreted as a single proximal policy improvement step (SPI), i.e., an implicit discretization of a manifold gradient flow induced by a critic-def

Read original source ↗ Open in workspace

recordType

paper

region

Global