SOURCE-LINKED INTELLIGENCE

3DWay: Generalizing Robot Manipulation via 3D Consistent Waypoints

arXiv · AI, language, vision and robotics · article · Sep 8, 2026 · UTC

Intermediate representations are key to bridging the modality gap between generalizable manipulation policies and large-scale pretrained vision-language models (VLMs). Among these, trajectory-based representations compactly represent motion-relevant cues, yet most existing approaches predict trajectories in 2D image space, resulting in intrinsic 3D ambiguity. Moreover, using 2D trajectories with depth still leaves the free-space waypoints ambiguous, limiting reliable 3D reasoning. To address this, we propose predicting 3D consistent waypoints (3DWay) from multi-view images. By reformulating 3D

Read original source ↗ Open in workspace

recordType

paper

region

Global