Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models

arXiv:2603.19233 · cs.RO · Submitted 2026-03-19 · Read on arXiv

cs.RO

Submitted: 2026-03-19

Updated: 2026-10-19

Comments: NeurIPS 2026 Poster, ICLR Multimodal Intelligence Workshop Oral, 46 pages, 14 figures

Project page: https://cwru-aism.github.io/vla-interp-page/ABSTRACT

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Fidelity — 2× in this paper · explained in 2 episodes
• Ablation — 1× in this paper · explained in 1 episode
• Benchmark — 1× in this paper · explained in 2 episodes
• Linear probe — 1× in this paper · explained in 2 episodes
• Multimodal Inputs — 1× in this paper · explained in 2 episodes
• Sparse Autoencoders (SAEs) — 1× in this paper · explained in 4 episodes
• Vision-Language-Action (VLA) Models — 1× in this paper · explained in 4 episodes

Sources

• Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
• Understanding intermediate layers using linear classifier probes
• ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperation
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
• $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
• RT-1: Robotics Transformer for Real-World Control at Scale
• GR-3 Technical Report
• Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
• Sparse Autoencoders Find Highly Interpretable Features in Language Models
• RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learning
• VITA-VLA: Efficiently Teaching Vision-Language Models to Act via Action Expert Distillation
• PaLM-E: An Embodied Multimodal Language Model
• AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation
• LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models
• Scaling and evaluating sparse autoencoders
• CAST: Counterfactual Labels Improve Instruction Following in Vision-Language-Action Models
• Pixels Versus Priors: Controlling Knowledge Priors in Vision-Language Models through Visual Counterfacts
• Mechanistic interpretability for steering vision-language-action models
• Steering CLIP's vision transformer with sparse autoencoders

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• ARC: A Reasoning Recipe for Robot Foundation Models
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies