FlowVLA: Enabling World Dynamics Understanding through Pluggable 3D Flow Learning
cs.RO
Submitted: 2026-10-12
Updated: 2026-10-12
Comments: 20 pages, 16 figures. Accepted at Conference on Robot Learning (CoRL) 2026
Project page: https://mmai-laboratory.github.io/flowvla
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recent advances in 3D reconstruction enable the extraction of temporally consistent 3D motion from RGB observations; however, current Vision-Language-Action (VLA) models still predominantly rely on
Abstract
Recent advances in 3D reconstruction enable the extraction of temporally consistent 3D motion from RGB observations; however, current Vision-Language-Action (VLA) models still predominantly rely on 2D image-text priors. Therefore, they do not accurately represent the fundamental dynamics of the 3D world associated with physical actions. This over-reliance on static 2D visual cues limits the capacity of VLA models in achieving robust robot manipulation. To address these limitations, we introduce FlowVLA, a reconstruction-guided, pluggable 3D flow learning framework aimed at empowering VLA models with an inherent understanding of action-relevant dynamics. Instead of requiring explicit 3D sensor inputs or altering the inference-time architecture, our method is designed to utilize temporally aligned 3D flow only during training. This allows the model to learn the dynamic evolution of the physical scene while preserving the computational efficiency of conventional 2D VLA backbones. To bridge scene-level dynamics with robotic manipulation, we propose a modular 3D flow learning framework with dual-level flow supervision. Specifically, point-level flow guides the model to forecast the future displacements of task-critical 3D points, while TCP-level flow aggregates these motions to align with the robot's TCP trajectories. This dual-level flow supervision encourages the model to focus on semantically meaningful dynamic regions, effectively coupling scene-level motion with precise control signals. Extensive evaluations across simulation and real-world experiments demonstrate that FlowVLA consistently improves manipulation performance, zero-shot robustness, and optimizes data efficiency. FlowVLA adds only lightweight auxiliary heads during training and discards them at inference. Therefore, FlowVLA introduces no additional 3D sensor inputs, architectural modifications, or inference-time computation.
Sources
- RT-1: Robotics Transformer for Real-World Control at Scale
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
- Robotic Control via Embodied Chain-of-Thought Reasoning
- Gemini Robotics: Bringing AI into the Physical World
- Octo: An Open-Source Generalist Robot Policy
- SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model
- BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Models
- 3D-VLA: A 3D Vision-Language-Action Generative World Model
- V-DPM: 4D Video Reconstruction with Dynamic Point Maps
- Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
- A Generalist Agent
- PaLM-E: An Embodied Multimodal Language Model
- Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation
- WorldVLA: Towards Autoregressive Action World Model
- An Embodied Generalist Agent in 3D World
- PolarNet: 3D Point Clouds for Language-Guided Robotic Manipulation
- LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness
- DM0: An Embodied-Native Vision-Language-Action Model towards Physical AI
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies