SOURCE-LINKED INTELLIGENCE

Mitigating Exploration Bias in RL for Multi-Instruction Following

arXiv · AI, language, vision and robotics · article · Aug 24, 2026 · UTC

RL has emerged as a powerful paradigm for enhancing the instruction following capabilities of LLMs. While existing training recipes achieve substantial gains, we find that they suffer from exploration bias towards easy instructions when the training data has multiple instructions in a prompt. This bias is caused by two main reasons: 1) the policy model's initial ability to satisfy hard instructions is too low to trigger successful exploration during RL training, so the optimization is biased towards easy instructions; and 2) canonical RL training recipes typically employ a cumulative reward (t

Read original source ↗ Open in workspace

recordType

paper

region

Global