SOURCE-LINKED INTELLIGENCE

Subspace Inference Enables Efficient Active Reward Learning from Preferences

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

Reinforcement learning from human feedback (RLHF) has emerged as a powerful yet sample-inefficient approach for learning reward models from human preferences, making active learning a critical component in synthesizing informative preference queries. However, effective uncertainty quantification required for active learning remains a key challenge for large neural network reward models. In this paper, we introduce PreferenceEKF, a sample-efficient approach that tracks reward model uncertainty by framing active preference learning as a sequential Bayesian filtering problem. Instead of relying o

Read original source ↗ Open in workspace

recordType

paper

region

Global