Box-WAM: Box Trajectories as Compact State Representations for World Action Models
cs.RO, cs.AI
Submitted: 2026-10-13
Updated: 2026-10-13
Code: https://github.com/nvidia-cosmos/cosmos-predict2
Project page: https://box-wam.github.io
License: http://creativecommons.org/licenses/by/4.0/
The gist: World action models (WAMs) are a category of policy models that ex- plicitly predict both actions and future world states.
Abstract
World action models (WAMs) are a category of policy models that ex- plicitly predict both actions and future world states. They have been shown to outperform action-only models across various robotic manipulation tasks. How- ever, representing the future with full-frame video prediction can be computation- ally expensive during both training and inference. This raises a central question: what compact state representation preserves the planning and supervision bene- fits of future prediction? In this paper, we propose BoxWAM, which replaces dense future frames with oriented bounding box trajectories of task-relevant ob- jects. By explicitly imagining object-centric future motion, our method provides a lightweight prediction target that captures the key physical changes induced by robot actions. We show that box trajectory prediction substantially reduces com- putational cost while improving action prediction performance, suggesting that object-centric compact states can serve as an efficient alternative to video-based world action modeling for robotic manipulation. Result videos can be found on https://box-wam.github.io
Sources
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- RT-1: Robotics Transformer for Real-World Control at Scale
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
- RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation
- OpenVLA: An Open-Source Vision-Language-Action Model
- EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos
- Motus: A Unified Latent Action World Model
- Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning
- World Action Models are Zero-shot Policies
- Fast-WAM: Do World Action Models Need Test-time Future Imagination?
- Causal World Modeling for Robot Control
- Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulation
- Video Generators are Robot Policies
- Unified Video Action Model
- Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation
- Large Video Planner Enables Generalizable Robot Control
- Mask World Model: Predicting What Matters for Robust Robot Policy Learning
- V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
- RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
- Dream to Control: Learning Behaviors by Latent Imagination
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies