From Sight to Foresight: Predictive Spatial Reasoning in Vision-Language Models
cs.CV
Submitted: 2026-10-02
Updated: 2026-10-02
Comments: Project page: https://brack-wang.github.io/spatialmind/
Project page: https://brack-wang.github.io/spatialmind
License: http://creativecommons.org/licenses/by/4.0/
The gist: Predicting future spatial states supports collision avoidance and timely decision-making in dynamic environments.
Terminology
Abstract
Predicting future spatial states supports collision avoidance and timely decision-making in dynamic environments. However, existing vision-language models (VLMs) and benchmarks for spatial reasoning primarily focus on observed scenes, leaving predictive spatial reasoning beyond the observed interval underexplored. To this end, we introduce SpatialMind, a metric-scale VLM for spatial reasoning and future prediction. Its metric depth adapter anchors spatial reasoning to real-world scale, while its progressive state chain establishes current spatial states and observed dynamics as the foundation for future prediction. Given a video prefix, SpatialMind predicts distances, motion directions, and spatial relations in both observed and unseen future frames. For training and evaluation, we build a scalable data engine that grounds entity descriptions in metric geometry to generate question-answer pairs and state supervision. Using this engine, we construct the SpatialMind-30K dataset and the SpatialMind-2K benchmark, both covering driving and everyday egocentric scenes. The benchmark spans eight tasks across three levels: current-state understanding, observed-dynamics understanding, and future prediction. Experiments show that SpatialMind substantially outperforms both general and spatially specialized models on our benchmark while achieving competitive zero-shot performance on VSI-Bench, OSI-Bench, and VLM4D.
Sources
- Cosmos 3: Omnimodal World Models for Physical AI
- Qwen3-VL Technical Report
- LIST3R: Long-sequence Instance-aware 3D Reconstruction
- LoRA: Low-Rank Adaptation of Large Language Models
- Reinforcing Dual-Path Reasoning in Spatial Vision Language Models
- Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction
- Depth Anything 3: Recovering the Visual Space from Any Views
- SpaceR: Reinforcing MLLMs in Video Spatial Reasoning
- Video-CoE: Reinforcing Video Event Prediction via Chain of Events
- RayMap3R: Inference-Time RayMap for Dynamic 3D Reconstruction
- CogniMap3D: Cognitive 3D Mapping and Rapid Retrieval
- VGGT-$\Omega$
- MLLM-4D: Towards Visual-based Spatial-Temporal Intelligence
- SCP: Spatial Causal Prediction in Video
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models