Spatial Latent Reasoning for Embodied Reference Understanding
cs.CV
Submitted: 2026-10-07
Updated: 2026-10-07
License: http://creativecommons.org/licenses/by/4.0/
The gist: Pointing-gesture visual grounding requires connecting hand geometry with the visual identity and extent of a referred object.
Terminology
Abstract
Pointing-gesture visual grounding requires connecting hand geometry with the visual identity and extent of a referred object. A central challenge for continuous latent reasoning is how to organize these complementary cues into useful intermediate supervision. We propose Spatial Latent Reasoning (SLR), a framework that structures this supervision around an ordered sequence of geometric and visual states. A spatial ray state is supervised by fingertip position and pointing direction, followed by four states aligned with target-region features. To construct the visual targets, we introduce parity pooling, which applies polyphase grouping to average region tokens on four interleaved spatial supports. All states are generated recurrently during training and inference; auxiliary annotations are required only during training. On EgoPoint-Ground, the framework improves mIoU over same-backbone supervised fine-tuning by 2.8, 17.5, and 21.1 percentage points on Qwen3.5-4B, Qwen2.5-VL-7B, and Qwen3-VL-8B, respectively, with improvements on both hard subsets. On YouRefIt, it achieves 77.6% precision at IoU 0.5, a numerical margin of 5.2 percentage points over the reported state of the art under differing evaluation protocols. Ablations support joint geometric and visual supervision on the standard and similar-object sets, and favor parity over three alternative pooling operators on the standard set. These results support task-structured supervision for continuous pointing grounding. We will release the code and supporting materials.
Sources
- Retrieve, Integrate, and Synthesize: Spatial-Semantic Grounded Latent Visual Reasoning
- AD-DINO: Attention-Dynamic DINO for Distance-Aware Embodied Reference Understanding
- ATLAS: Agentic or Latent Visual Reasoning? One Word is Enough for Both
- Training Large Language Models to Reason in a Continuous Latent Space
- Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
- PointVG-R: Internalizing Geometric Reasoning in MLLMs for Precise Pointing Localization via Visual Chain of Thought
- DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models