Urbex: Agentic Spatial Grounding for City-Scale 3D Scenes
cs.CV, cs.AI
Submitted: 2026-10-13
Updated: 2026-10-13
Comments: NeurIPS 2026. 23 pages, 10 figures. Code: https://github.com/AMAP-ML/Urbex
Code: https://github.com/AMAP-ML/Urbex
License: http://creativecommons.org/licenses/by/4.0/
The gist: City-scale 3D grounding is commonly formulated as candidate ranking, which requires precomputed instance proposals and becomes inefficient in large outdoor scenes.
Terminology
Abstract
City-scale 3D grounding is commonly formulated as candidate ranking, which requires precomputed instance proposals and becomes inefficient in large outdoor scenes. We propose Urbex, an agentic vision-language framework that instead casts grounding as active spatial search. Urbex represents a city-scale 3D scene as an interactive multi-view environment, allowing a VLM agent to query landmarks, zoom into local regions, render oblique views, and commit a grounded bounding box. We optimize this tool-use policy with reinforcement learning on CityRefer, using smooth localization rewards and lightweight shaping rewards for efficient exploration. Experiments show that Urbex achieves the best Hit@1 and IoU on CityRefer, supports candidate-free grounding without ground-truth boxes, and is substantially faster than the strongest city-scale ranking baseline. Zero-shot results on STPLS3D-Refer further indicate improved cross-scene robustness.
Sources
- GPT-4 Technical Report
- Qwen3-VL Technical Report
- Language-Guided 3D Object Detection in Point Cloud for Autonomous Driving
- MobileVLM V2: Faster and Stronger Baseline for Vision Language Model
- OpenGround: Planning-based Online Perception for Open-World 3D Visual Grounding
- Thinking with Map: Reinforced Parallel Map-Augmented Agent for Geolocalization
- 3EED: Ground Everything Everywhere in 3D
- View-on-Graph: Zero-shot 3D Visual Grounding via Vision-Language Reasoning on Scene Graphs
- SkillClaw: Let Skills Evolve Collectively with Agentic Evolver
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Think, Act, Build: An Agentic Framework with Vision Language Models for Zero-Shot 3D Visual Grounding
- LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineering
- UniGround: Universal 3D Visual Grounding via Training-Free Scene Parsing
- Task-oriented Sequential Grounding and Navigation in 3D Scenes
- Code2World: A GUI World Model via Renderable Code Generation
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models