From Pixels to Executable Physics: Behavior-Grounded World Models for Embodied Action Consequence Prediction

arXiv:2610.17630 · cs.RO, cs.CV · Submitted 2026-10-13 · Read on arXiv

cs.RO, cs.CV

Submitted: 2026-10-13

Updated: 2026-10-13

Code: https://github.com/Genesis-Embodied-AI/Genesis

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• World Model — 1× in this paper · explained in 7 episodes

Sources

• TuneNet: One-Shot Residual Tuning for System Identification and Sim-to-Real Robot Task Transfer
• Walk through Paintings: Egocentric World Models from Internet Priors
• Navigation World Models
• Genie: Generative Interactive Environments
• Emerging Properties in Self-Supervised Vision Transformers
• Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience
• Identifying Terrain Physical Parameters from Vision -- Towards Physical-Parameter-Aware Locomotion and Navigation
• Dream to Control: Learning Behaviors by Latent Imagination
• Learning Latent Dynamics for Planning from Pixels
• Excavating in the Wild: The GOOSE-Ex Dataset for Semantic Segmentation
• LoRA: Low-Rank Adaptation of Large Language Models
• VBench: Comprehensive Benchmark Suite for Video Generative Models
• RELLIS-3D Dataset: Data, Benchmarks and Analysis
• WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actions
• Cosmos 3: Omnimodal World Models for Physical AI
• DINOv2: Learning Robust Visual Features without Supervision
• BayesSim: adaptive domain randomization via probabilistic inference for robotics simulators
• Sampling-Based System Identification with Active Exploration for Legged Robot Sim2Real Learning
• Wan: Open and Advanced Large-Scale Video Generative Models
• ActiveSAM: Fast and Accurate Open-Vocabulary Semantic Segmentation with Frozen SAM 3

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• ARC: A Reasoning Recipe for Robot Foundation Models
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies