Urbex: Agentic Spatial Grounding for City-Scale 3D Scenes

arXiv:2610.18356 · cs.CV, cs.AI · Submitted 2026-10-13 · Read on arXiv

cs.CV, cs.AI

Submitted: 2026-10-13

Updated: 2026-10-13

Comments: NeurIPS 2026. 23 pages, 10 figures. Code: https://github.com/AMAP-ML/Urbex

Code: https://github.com/AMAP-ML/Urbex

License: http://creativecommons.org/licenses/by/4.0/

The gist: City-scale 3D grounding is commonly formulated as candidate ranking, which requires precomputed instance proposals and becomes inefficient in large outdoor scenes.

Terminology

Abstract

City-scale 3D grounding is commonly formulated as candidate ranking, which requires precomputed instance proposals and becomes inefficient in large outdoor scenes. We propose Urbex, an agentic vision-language framework that instead casts grounding as active spatial search. Urbex represents a city-scale 3D scene as an interactive multi-view environment, allowing a VLM agent to query landmarks, zoom into local regions, render oblique views, and commit a grounded bounding box. We optimize this tool-use policy with reinforcement learning on CityRefer, using smooth localization rewards and lightweight shaping rewards for efficient exploration. Experiments show that Urbex achieves the best Hit@1 and IoU on CityRefer, supports candidate-free grounding without ground-truth boxes, and is substantially faster than the strongest city-scale ranking baseline. Zero-shot results on STPLS3D-Refer further indicate improved cross-scene robustness.

Sources

Related papers