Thinker-WM: A Unified World Model with Shared Multimodal Representations for Embodied Intelligence
cs.AI, cs.CV, cs.RO
Submitted: 2026-10-13
Updated: 2026-10-19
Comments: Project page: https://thinkercosmos.ubtrobot.com/thinker-wm
Code: https://github.com/XiaomiRobotics/XiaomiRobotics-U0
Project page: https://thinkercosmos.ubtrobot.com/thinker-wm
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: World models have emerged as a promising paradigm for embodied AI, yet existing systems often remain fragmented in the modalities they model and the functions they support.
Terminology
Abstract
World models have emerged as a promising paradigm for embodied AI, yet existing systems often remain fragmented in the modalities they model and the functions they support. In particular, visual dynamics, geometric structure, and executable action are commonly coupled through separate interfaces or downstream modules rather than being jointly represented and evolved within a shared dynamics model. In this work, we present Thinker-WM, a unified multimodal world model that jointly models and generates RGB, metric depth, and robot action through a shared intermediate Unified Dynamics Transformer with modality-specific encoders and decoders. The model performs multimodal unification at the representation level: appearance, geometry, and action are transformed into a common dynamic representation and jointly evolved by shared parameters, while their distinct output structures are preserved. The same parameterization supports multiple functions, including joint video-action generation, world prediction, policy inference, action-conditioned generation, video-to-action generation, and inverse dynamics. To examine whether the shared representation captures action-related information, we introduce a lightweight motion probe that shows the frozen visual representation contains recoverable motion information. Thinker-WM is trained on a heterogeneous corpus comprising approximately 20K hours of egocentric video, 10K hours of real-robot data, and 1K hours of simulation data, including approximately 4K hours of real-robot data with aligned depth supervision. Extensive experiments demonstrate strong performance across embodied world-model evaluations. Thinker-WM ranks first on EWMBench, WorldOlympiad and WorldArena, while controlled studies further demonstrate the benefits of action conditioning, geometric interaction, and multi-function training within the shared representation.
Sources
- Qwen3-VL Technical Report
- Motus: A Unified Latent Action World Model
- World Models
- Mastering Diverse Domains through World Models
- GAIA-1: A Generative World Model for Autonomous Driving
- EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models
- EgoMimic: Scaling Imitation Learning via Egocentric Video
- Diffusion ReRoll: Revisable Denoising for Robotic Sequential Prediction
- Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model
- WAM4D: Fast 4D World Action Model via Spatial Register Tokens
- Depth Anything 3: Recovering the Visual Space from Any Views
- GeoSem-WAM: Geometry- and Semantic-Aware World Action Models
- Motubrain: An Advanced World Action Model for Robot Control
- RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)
- OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation
- Cosmos World Foundation Model Platform for Physical AI
- Cosmos 3: Omnimodal World Models for Physical AI
- EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World
- WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models
- Wan: Open and Advanced Large-Scale Video Generative Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems