VLA-GAUGE: Systematically Evaluating Generalization and Robustness in Vision-Language-Action Models

arXiv:2610.16567 · cs.CV, cs.AI, cs.RO · Submitted 2026-10-12 · Read on arXiv

cs.CV, cs.AI, cs.RO

Submitted: 2026-10-12

Updated: 2026-10-12

Comments: 18 pages, 8 figures, 4 tables

Project page: https://vla-gaugesite.pages.dev

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Generalization — 1× in this paper · explained in 13 episodes
• Robustness — 1× in this paper · explained in 5 episodes

Sources

• RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
• PaLM-E: An Embodied Multimodal Language Model
• OpenVLA: An Open-Source Vision-Language-Action Model
• LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization
• LIBERO-X: Robustness Litmus for Vision-Language-Action Models
• THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation
• Beyond Single-Axis Testing: Paired Evaluation of Compound Robustness in Vision-Language-Action Policies
• Evaluating Real-World Robot Manipulation Policies in Simulation
• VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Models
• $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
• RoboHarness: A Memory-Augmented Policy Harness for Vision-Language-Action Model Robustness via In-Context Adaptation
• UniVLA: Learning to Act Anywhere with Task-centric Latent Actions
• NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Tasks
• FAST: Efficient Action Tokenization for Vision-Language-Action Models
• DeepThinkVLA: Enhancing Reasoning Capability of Vision-Language-Action Models
• SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning
• Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
• Interactive Post-Training for Vision-Language-Action Models
• CronusVLA: Towards Efficient and Robust Manipulation via Multi-Frame Vision-Language-Action Modeling
• $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models