SOURCE-LINKED INTELLIGENCE

FlowCPO: A Unified Divergence View of Preference Alignment for Flow Models

arXiv · AI, language, vision and robotics · article · Sep 9, 2026 · UTC

Preference alignment for flow and diffusion models now spans online reinforcement learning and offline preference optimization, but the relation between these methods remains unclear. In particular, existing forward-process alignment methods require fresh samples from the current model, while offline methods based on fixed preference pairs rely primarily on positive-only fine-tuning or DPO-style likelihood-ratio surrogates. We organize these approaches through a divergence-based framework and introduce FlowCPO, an offline forward-KL objective that uses both preferred and dispreferred samples w

Read original source ↗ Open in workspace

recordType

paper

region

Global