DexFLEX: Contact-Aware Foundation Controller for Command-Guided Dexterity
cs.RO, cs.AI, cs.LG
Submitted: 2026-10-09
Updated: 2026-10-09
Code: https://github.com/kevinzakka/mink
License: http://creativecommons.org/licenses/by/4.0/
The gist: Dexterous robot hands can receive useful motion intent from teleoperation or learned policies, but successful execution also depends on contact decisions that these commands rarely specify.
Abstract
Dexterous robot hands can receive useful motion intent from teleoperation or learned policies, but successful execution also depends on contact decisions that these commands rarely specify. We introduce DexFLEX, a contact-aware foundation controller that turns upstream fingertip-motion drafts into contact-consistent joint commands. Instead of treating a draft as a trajectory to copy, DexFLEX treats it as evidence about intent: from the current tactile-proprioceptive state, it proposes feasible short-horizon motion chunks, predicts their contact consequences, and selects the candidate that best follows the command while preserving future contact stability. At inference time, a simple Draft-Dream-Select procedure combines pure-prior proposals for recovery with draft-seeded proposals for responsiveness, then decodes the selected motion into executable joint-space targets. Across degraded-command simulation, real-world shared control, and visuomotor policy learning, the same trained controller improves robustness without retraining, increasing real-world teleoperation success from 29.2% to 78.3% and reaching 46.7% policy-learning success, 3.5 times direct joint-action prediction, and 1.56 times prior-only correction. Videos are available at dex-flex.github.io.
Sources
- Shared Autonomy via Deep Reinforcement Learning
- Residual Policy Learning for Shared Autonomy
- To the Noise and Back: Diffusion for Shared Autonomy
- DexterityGen: Foundation Controller for Unprecedented Dexterity
- Rotating without Seeing: Towards In-hand Dexterity through Touch
- Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play
- RT-1: Robotics Transformer for Real-World Control at Scale
- Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulation
- OmniVTA: Visuo-Tactile World Modeling for Contact-Rich Robotic Manipulation
- Visuo-Tactile World Models
- VTAM: Video-Tactile-Action Models for Complex Physical Interaction Beyond VLAs
- Learning Versatile Humanoid Manipulation with Touch Dreaming
- AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System
- DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation
- SPIDER: Scalable Physics-Informed Dexterous Retargeting
- FingerEye: Learning Dexterous Manipulation with Continuous Vision-Tactile Sensing
- ManiFlow: A General Robot Manipulation Policy via Consistency Flow Training
- SimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulation
- 3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing
- FlexiTac: A Low-Cost, Open-Source, Scalable Tactile Sensing Solution for Robotic Systems
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies