SOURCE-LINKED INTELLIGENCE
A Zeroth-Order Paradigm for LLM Preference Alignment
Direct preference alignment methods are widely used to align large language models (LLMs) with human preferences because of their computational and memory efficiency. However, likelihood displacement motivates alternative ways to extract information from preference pairs with small likelihood margins. In this paper, we propose and analyze Comparison-based Preference Optimization (ComPO), a zeroth-order alignment method based on comparison oracles. ComPO extracts directional information from these pairs without directly optimizing a differentiable preference loss on them. We establish a converg
Read original source ↗ Open in workspace
- recordType
- paper
- region
- Global
Evidence & attribution
- arXiv · AI, language, vision and robotics · 2026-09-16T17:59:35.000Z
- arXiv · Artificial Intelligence · 2026-09-16T17:59:35.000Z
First collected: 2026-09-19T20:26:32.566Z. This is not the publication date.