HarnessWAM: Bridging Prediction and Deliberation in World Action Models
cs.RO
Submitted: 2026-08-10
Updated: 2026-10-10
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: World Action Models (WAMs) jointly learn environmental dynamics and robot actions, introducing priors over physical evolution into embodied control.
Terminology
Abstract
World Action Models (WAMs) jointly learn environmental dynamics and robot actions, introducing priors over physical evolution into embodied control. However, finite-horizon prediction and action generation are insufficient for complex embodied tasks that require global planning, cross-stage state maintenance, execution verification, and failure recovery. We refer to this mismatch as the prediction-deliberation gap. To address this gap, we propose HarnessWAM, an agentic framework for WAMs. HarnessWAM employs a vision-language-model-based Task Manager to maintain an evidence-grounded scene belief and organize the global instruction into a structured task graph. Capability-conditioned executable-space projection then compiles the graph into WAM-supported skill sequences that satisfy task dependencies and embodiment-state constraints. During execution, HarnessWAM couples a lightweight progress estimator with event-triggered Task Manager deliberation. The estimator supplies frequent execution cues, while the Task Manager uses observations, task state, and interaction history to verify outcomes and decide whether to advance, acquire evidence, revise the plan, or recover. Local recovery preserves acquired scene knowledge so that execution can resume after failure. HarnessWAM achieves state-of-the-art average performance on RoboMemArena, with full-task and subtask success rates of 64.4% and 76.8%, respectively, and reaches 23.7% success on RoboCerebra Ideal. On a physical UR3 arm, full-task success reaches 95% for shape-conditioned tabletop tidying and 90% for memory-dependent box exploration. These results demonstrate the effectiveness of model-external agentic orchestration in bridging the prediction-deliberation gap for complex embodied tasks.
Sources
- Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress
- Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
- SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulation
- RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agents
- RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies
- Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control
- Inner Monologue: Embodied Reasoning through Planning with Language Models
- VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- OpenVLA: An Open-Source Vision-Language-Action Model
- RoboMemArena: A Comprehensive and Challenging Robotic Memory Benchmark
- Causal World Modeling for Robot Control
- Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation
- SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning
- HiMem-WAM: Hierarchical Memory-Gated World Action Models for Robotic Manipulation
- MemoryWAM: Efficient World Action Modeling with Persistent Memory
- World Action Models are Zero-shot Policies
- Fast-WAM: Do World Action Models Need Test-time Future Imagination?
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies