KPGrasp: Scalable Keypoint Flow Matching for Dexterous Grasp Generation
cs.RO
Submitted: 2026-06-08
Updated: 2026-10-10
Comments: 14 pages, 7 figures, 6 tables; Accepted by CoRL 2026
Code: https://github.com/kevinzakka/mink
License: http://creativecommons.org/licenses/by/4.0/
The gist: Generating high-quality dexterous grasps remains challenging for learning-based methods, which often depend on carefully tuned contact losses or costly contact-based test-time refinement.
Terminology
Abstract
Generating high-quality dexterous grasps remains challenging for learning-based methods, which often depend on carefully tuned contact losses or costly contact-based test-time refinement. We present KPGrasp, a flow-matching framework that learns dexterous grasp priors from large-scale data rather than relying on contact losses or contact-based test-time refinement. KPGrasp couples an all-Euclidean 3D hand-keypoint parameterization with a simple yet scalable Transformer flow model. The parameterization avoids the drawbacks of the conventional mixed SE(3) pose and joint-angle output space, expresses grasps in the same frame as the object point cloud, and thus enables native spatial reasoning; the Transformer flow model is trained with only the standard flow-matching loss and scales effectively with data, model capacity, and batch size. Experiments demonstrate state-of-the-art performance on two simulation benchmarks. On the Dexonomy benchmark, it reaches a 76.3% grasp success rate, improving over the strongest directly comparable baseline by 47.4% while reducing penetration depth to 2.4 mm. The same model also achieves the best average performance on the DexGrasp Anything benchmark without fine-tuning. For batched inference, KPGrasp requires only 0.032 s per grasp. Finally, real-world experiments on 20 diverse objects demonstrate that the pipeline can be deployed in a real-world setup. Project Page: https://yuansen-huang.github.io/KPGrasp-page/
Sources
- SpringGrasp: Synthesizing Compliant, Dexterous Grasps under Shape Uncertainty
- Dexonomy: Synthesizing All Dexterous Grasp Types in a Grasp Taxonomy
- DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation
- Dex1B: Learning with 1B Demonstrations for Dexterous Manipulation
- GenDexGrasp: Generalizable Dexterous Grasping
- Deep Differentiable Grasp Planner for High-DOF Grippers
- Learning Robust Real-World Dexterous Grasping Policies via Implicit Shape Augmentation
- FFHFlow: Diverse and Uncertainty-Aware Dexterous Grasp Generation via Flow Variational Inference
- $\mathcal{D(R,O)}$ Grasp: A Unified Representation of Robot and Object Interaction for Cross-Embodiment Dexterous Grasping
- HOGraspFlow: Taxonomy-Aware Hand-Object Retargeting for Multi-Modal SE(3) Grasp Generation
- SE(3)-Equivariant Diffusion Policy in Spherical Fourier Space
- Point Policy: Unifying Observations and Actions with Key Points for Robot Manipulation
- World Models for Learning Dexterous Hand-Object Interactions from Human Videos
- Dexterity from Smart Lenses: Multi-Fingered Robot Manipulation with In-the-Wild Human Demonstrations
- Flow Matching for Generative Modeling
- FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
- Score-Based Generative Modeling through Stochastic Differential Equations
- Muon is Scalable for LLM Training
- RealDex: Towards Human-like Grasping for Robotic Dexterous Hand
- Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies