SOURCE-LINKED INTELLIGENCE

Speak to the City: Multimodal Resolution for Outside-the-Vehicle References

arXiv · AI, language, vision and robotics · article · Sep 13, 2026 · UTC

As autonomous vehicles and Extended Reality (XR) headsets enable novel in-car interactions, seamlessly querying physical landmarks, known as Outside-the-Vehicle Referencing (OVR), remains challenging due to ego-motion and referential ambiguity. We present a robust, multimodal OVR framework fusing user gaze and natural language to identify Points of Interest (POIs). To address the scarcity of dynamic vehicular data, we developed a VR-based pipeline synchronizing 360-degree transit videos with vehicle GNSS telemetry. Through a user study (N=46) mapping passenger head orientation into a 3D geospa

Read original source ↗ Open in workspace

recordType

paper

region

Global