SOURCE-LINKED INTELLIGENCE

Don't Mask the Environment: Observation Supervision Changes How Agents Explore Under RL

arXiv · Artificial Intelligence · article · Sep 17, 2026 · UTC

Agent trajectories record what an agent does and what happens next. Yet standard supervised fine-tuning (SFT) applies loss only to agent-authored action tokens, using environment observations as context but not as prediction targets. We ask whether this convention provides the best initialization for subsequent reinforcement learning. We introduce ActObs, which also supervises the observation tokens already present in each trajectory. Although deployed agents never generate observations, learning to predict them encourages the policy to model action consequences without adding data, parameters

Read original source ↗ Open in workspace

recordType

paper

region

Global