Exploring the Limits of Combinatorial Generalization in Embodied Manipulation
cs.RO
Submitted: 2026-10-11
Updated: 2026-10-11
Code: https://github.com/AutoLab-SAI-SJTU/RoboCombine
License: http://creativecommons.org/licenses/by/4.0/
The gist: In embodied manipulation, Vision Language Action (VLA) models can execute atomic skills, but long-horizon manipulation requires recombining these skills without demonstrations for every skill
Terminology
Abstract
In embodied manipulation, Vision Language Action (VLA) models can execute atomic skills, but long-horizon manipulation requires recombining these skills without demonstrations for every skill combination. The full instruction route treats skill chaining as long-horizon imitation. In a controlled study, full instruction training on limited skill combination sequences reaches 40.0% ordered success on covered combinations but only 7.7% on recombination, suggesting memorization of sequence templates rather than robust skill recombination. We introduce Hierarchical Subtask Execution (HSE), a diagnostic framework that exposes skill recombination through a shared modular interface over the planner, trigger, and policy, decomposing long-horizon failures into separable modes. We introduce RoboCombine, a benchmark for skill recombination without combined skill training trajectories, with metrics for ordered success and transition conversion across subtask boundaries. RoboCombine reveals that atomic competence alone does not ensure ordered execution. Failures concentrate at transitions where downstream subtasks launch from handoff states from preceding subtasks rather than canonical atomic starts. We identify this mismatch as the start state gap. To address it, we introduce Pose Robust Task Initialization (PRTI), a lightweight atomic task post-training strategy that broadens start state coverage beyond canonical starts. Under oracle timed HSE on RoboCombine, PRTI reaches 18.16% ordered success, compared with 4.21% under random perturbation and 9.47% under a robot-reset oracle, without combined skill training trajectories. These results show that combinatorial generalization requires handoff robust atomic skills that convert first stage progress into reliable downstream execution, not merely decomposition into known subtasks. Code: https://github.com/AutoLab-SAI-SJTU/RoboCombine.
Sources
- Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
- Motus: A Unified Latent Action World Model
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
- RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
- Efficient Bimanual Manipulation Using Learned Task Schemas
- Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks
- Hierarchical Variational Imitation Learning of Control Programs
- Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planning
- CoPa: General Robotic Manipulation through Spatial Constraints of Parts with Foundation Models
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- OpenVLA: An Open-Source Vision-Language-Action Model
- MolmoAct: Action Reasoning Models that can Reason in Space
- OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoning
- LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
- HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model
- CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks
- FAST: Efficient Action Tokenization for Vision-Language-Action Models
- SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model
- Efficient Diffusion Transformer Policies with Mixture of Expert Denoisers for Multitask Learning
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies