SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models
Jing Wu, Jianhua Wu, Jiayi Guan, Jiahong Chen, Jinghui Lu, Hangjun Ye, Bingzhao Gao, Long Chen
cs.CV, cs.CL, cs.LG
Submitted: 2026-08-03
Comments: 27 pages,13 figures,16 tables
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Vision-Language Models (VLMs) perform well on commonsense reasoning tasks but struggle with visual spatial reasoning.
Terminology
Abstract
Vision-Language Models (VLMs) perform well on commonsense reasoning tasks but struggle with visual spatial reasoning. Most existing solutions introduce extra 3D prior inputs or external spatial encoders, which increase complexity and degrade the underlying VLMs' general-purpose capabilities after spatial fine-tuning. To this end, we propose a parameter-efficient Spatio-vision Language Models (SpatioLM), that enhances spatial intelligence without extra 3D prior inputs or third-party spatial encoders. Concretely, we design a plug-and-play and non-invasive spatio-vision module that elicits the spatial knowledge inherent in VLMs. Furthermore, we innovatively leverage pseudo depth and camera information as supervision to guide the model in learning physically coherent representations. Extensive experiments show that SpatioLM achieves significant improvements in diverse tasks, including spatial perception and understanding while effectively limiting the degradation of general capabilities. Notably, the model achieves an impressive score of 71.6 on the VSI-Bench (the first model to surpass 70). In addition, it attains competitive performance when transferred to embodied manipulation tasks. Code is available at spatio-lm.
Sources
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- Depth Pro: Sharp Monocular Metric Depth in Less Than a Second
- UniVLA: Learning to Act Anywhere with Task-centric Latent Actions
- Virtual KITTI 2
- Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning
- Qwen3-VL Technical Report
- Scaling Spatial Intelligence with Multimodal Foundation Models
- DepthLM: Metric Depth From Vision Language Models
- Qwen2.5-VL Technical Report
- ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data
- Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas
- Reasoning in Space via Grounding in the World
- MiMo-Embodied: X-Embodied Foundation Model Technical Report
- Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videos
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- 3DRS: MLLMs Need 3D-Aware Representation Supervision for Scene Understanding
- GPT-4o System Card
- VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction
- OpenVLA: An Open-Source Vision-Language-Action Model
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models