GeoThinker- F: Endogenous Predictive World Representations for Fast Spatial Intelligence
cs.CV
Submitted: 2026-10-14
Updated: 2026-10-14
Code: https://github.com/Li-Hao-yuan/GeoThinker
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Existing geometry-aware Multimodal Large Language Models (MLLMs) increasingly rely on external 3D encoders to provide structural priors, but executing these additional encoder paths at inference
Abstract
Existing geometry-aware Multimodal Large Language Models (MLLMs) increasingly rely on external 3D encoders to provide structural priors, but executing these additional encoder paths at inference introduces substantial latency and computational overhead. We propose GeoThinker- F, an efficient framework that internalizes spatiotemporal perception by transforming causal MLLM hidden states into predictive latent world representations. Our key insight is that intermediate semantic representations contain sufficient information to predict useful representations of current geometry and future semantics, without requiring an exact reproduction of features from a dedicated 3D encoder. To realize this idea, GeoThinker- F adopts a decoupled two-stage training strategy. In the first stage, the MLLM backbone is frozen while lightweight Latent Frame Prediction (LFP) heads learn from an offline 3D geometry expert and next-frame semantic targets. In the second stage, the external expert is removed, and the model is jointly optimized to generate and selectively integrate its self-predicted priors through a specialized Spatial-Grounded Temporal (SGT) Fusion module. Extensive evaluations show that GeoThinker- F eliminates runtime dependence on external 3D geometry encoders while maintaining competitive performance across diverse spatial intelligence benchmarks. It achieves up to a 3.6 times inference speedup, reducing latency from 1.45, s to 0.40, s while retaining strong spatial reasoning performance.
Sources
- V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
- Benchmark Designers Should "Train on the Test Set" to Expose Exploitable Non-Visual Shortcuts
- Holistic Evaluation of Multimodal LLMs on Spatial Intelligence
- VGGT-Det: Mining VGGT Internal Priors for Sensor-Geometry-Free Multi-View Indoor 3D Object Detection
- Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Emerging Properties in Unified Multimodal Pretraining
- Gemini: A Family of Highly Capable Multimodal Models
- Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
- GPT-4o System Card
- LLaVA-OneVision: Easy Visual Task Transfer
- ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models
- Thinking with Geometry: Active Geometry Integration for Spatial Reasoning
- ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving
- MMSI-Video-Bench: A Holistic Benchmark for Video-Based Spatial Intelligence
- LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
- Qwen2.5-VL Technical Report
- Qwen3 Technical Report
- StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing
- Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awareness
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models