SOURCE-LINKED INTELLIGENCE

Direct Diversity Optimization for Diverse Successful Trajectories in Preference Post-Training

arXiv · AI, language, vision and robotics · article · Sep 9, 2026 · UTC

LLM agents for sequential decision tasks are often post-trained with trajectory-level outcome labels, but such labels provide little supervision for preserving multiple successful branches from the same decision state. We study this problem as successful strategy coverage: how broadly a model realizes distinct successful strategies under a fixed rollout budget. We present Direct Diversity Optimization (DDO), an offline post-training method that combines Divergence-Tree Collection (DTC) with the Reference-Relative Target-Odds Objective (RTO). DTC constructs state-aligned branch sets rooted at s

Read original source ↗ Open in workspace

recordType

paper

region

Global