The Human--VLA Communication Gap: What Humans Leave Implicit and What Policies Cannot Recover
cs.RO
Submitted: 2026-10-13
Updated: 2026-10-13
Project page: https://commgap.github.io
License: http://creativecommons.org/licenses/by/4.0/
The gist: A robot may know how to perform a task yet fail when a human asks for it naturally.
Abstract
A robot may know how to perform a task yet fail when a human asks for it naturally. We study this human--VLA communication gap by separating an information gap, where task semantics, grounding, or event structure remain unresolved, from a residual execution gap, where even a fully specified task depends on how it is organized and invoked. To do this, we characterize how people naturally express demonstrated objectives and evaluate controlled variants of the same physical tasks, with detailed analysis on π 0.5 and consistent trends on π 0-FAST and Cosmos3-Edge. Our study reveals that humans increasingly compress instructions as tasks grow, relying more on contextual grounding and implicit execution structure. Current policies are misaligned with this behavior: they tolerate grounding dependence relatively well, but struggle when task semantics or event structure must be reconstructed. Even after the information gap is resolved, execution remains sensitive to command granularity and invocation state. These findings identify concrete targets for human-centered policy learning and interfaces that better bridge natural human objectives to robot behavior. Project website: https://commgap.github.io/.
Sources
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- FAST: Efficient Action Tokenization for Vision-Language-Action Models
- Cosmos 3: Omnimodal World Models for Physical AI
- LIBERO-Para: A Diagnostic Benchmark and Metrics for Paraphrase Robustness in VLA Models
- LangGap: Diagnosing and Closing the Language Gap in Vision-Language-Action Models
- From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Models
- Grounded Semantic Re-Binding for Robust Instruction Generalization in Vision-Language-Action Models
- Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detection
- SAM 2: Segment Anything in Images and Videos
- Qwen2.5-VL Technical Report
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies