SOURCE-LINKED INTELLIGENCE

Open-vocabulary 3D object detection with promptable segmentation

arXiv · AI, language, vision and robotics · article · Sep 16, 2026 · UTC

Three-dimensional object detection for autonomous driving is dominated by detectors trained on large corpora of human-annotated 3D boxes. Such a detector learns a fixed category list, and everything outside it is invisible. This paper asks whether the task can be solved training-free and open-vocabulary. A promptable segmentation model (SAM3), queried with class names as text prompts, supplies instance masks in the vehicle's six surround-view cameras, and the masks are turned into metric 3D boxes using the geometry of the scene. The core is a controlled three-stage comparison on nuScenes in wh

Read original source ↗ Open in workspace

recordType

paper

region

Global