Artificial Foveated Perception for Mitigating Shortcut Learning in Robotic Foundation Models

arXiv:2607.10655 · cs.RO · Submitted 2026-07-12 · Read on arXiv

cs.RO

Submitted: 2026-07-12

Updated: 2026-10-12

Comments: Accepted to CoRL 2026

Code: https://github.com/Apollo-Lab-Yale/afp-annotation-tool

Project page: https://apollo-lab-yale.github.io/26-CoRL-AFP-website

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Foundation Model — 5× in this paper · explained in 3 episodes
• Foundation Models — 5× in this paper · explained in 7 episodes
• Fine-Tuning — 3× in this paper · explained in 5 episodes
• Artificial Foveated Perception — 2× in this paper
• Grounding — 2× in this paper · explained in 3 episodes
• Ablation — 1× in this paper · explained in 1 episode
• Core — 1× in this paper · explained in 2 episodes
• Cross-Embodiment Transfer — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 13 episodes

Sources

• $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
• SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
• GPT-4 Technical Report
• What Can RL Bring to VLA Generalization? An Empirical Study
• Enhancing Generalization in Vision-Language-Action Models by Preserving Pretrained Representations
• Dynamic Rank Adjustment in Diffusion Policies for Efficient and Flexible Training
• Don't Blind Your VLA: Aligning Visual Representations for OOD Generalization
• Gaze-Guided Learning: Avoiding Shortcut Bias in Visual Classification
• SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
• PaliGemma: A versatile 3B VLM for transfer
• A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation
• Motus: A Unified Latent Action World Model
• Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning
• Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better
• Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
• PRISM-DP: Spatial Pose-based Observations for Diffusion-Policies via Segmentation, Mesh Generation, and Pose Tracking
• Hybrid Diffusion Policies with Projective Geometric Algebra for Efficient Robot Manipulation Learning
• Optimizing Active Perception for Learning Simultaneous Viewpoint Selection and Manipulation with Diffusion Policy
• Navigating Shortcuts, Spurious Correlations, and Confounders: From Origins via Detection to Mitigation
• Invariant Risk Minimization

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• ARC: A Reasoning Recipe for Robot Foundation Models
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies