OpenViTac: Learning and Benchmarking Visuo-Tactile Policies in a Unified Sim-and-Real Framework
cs.RO
Submitted: 2026-10-07
Updated: 2026-10-07
Comments: Project website: https://fvl-repo.github.io/OpenViTac/
Project page: https://fvl-repo.github.io/OpenViTac
License: http://creativecommons.org/licenses/by/4.0/
The gist: Tactile feedback provides embodied agents with physical information beyond visual observations, enabling more reliable interaction with the real world.
Terminology
Abstract
Tactile feedback provides embodied agents with physical information beyond visual observations, enabling more reliable interaction with the real world. However, despite the rapid progress of vision-tactile-language-action (VTLA) policies, there remains a lack of unified benchmarks for evaluating tactile-enabled robot manipulation across simulation and the real world. To address this gap, we introduce OpenViTac, a visuo-tactile manipulation benchmark for evaluating robot policies across simulation and the real world. OpenViTac organizes contact-rich manipulation into four tactile-relevant capability dimensions and provides paired simulation-real-world settings for consistent evaluation of VLA, WAM, and VTLA policies. Building upon this benchmark, we investigate how different tactile representations and integration strategies affect the performance of pretrained VLA models. Correspondingly, we introduce OpenVTLA, a tactile augmentation framework that combines the best-performing representation and integration strategy. Furthermore, we leverage the paired benchmark setting to study sim-real co-training and analyze factors affecting cross-domain policy learning. Together, OpenViTac provides a unified platform for evaluating and advancing visuo-tactile robot manipulation.
Sources
- World Simulation with Video Foundation Models for Physical AI
- UniVTAC: A Unified Simulation Platform for Visuo-Tactile Manipulation Data Generation, Learning, and Benchmarking
- RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
- RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies
- Visuo-Tactile Transformers for Manipulation
- StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing
- AnyTouch 2: General Optical Tactile Representation Learning For Dynamic Tactile Perception
- Tactile-VLA: Unlocking Vision-Language-Action Model's Physical Knowledge for Tactile Generalization
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- Self-Supervised Visuo-Tactile Pretraining to Locate and Follow Garment Features
- A Mechanistic Analysis of Sim-and-Real Co-Training in Generative Robot Policies
- Causal World Modeling for Robot Control
- Evaluating Real-World Robot Manipulation Policies in Simulation
- Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation
- ManiFeel: Benchmarking and Understanding Visuotactile Manipulation Policy Learning
- InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action for Compositional Generalization
- Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation
- Tactile Modality Fusion for Vision-Language-Action Models
- TacEx: GelSight Tactile Simulation in Isaac Sim -- Combining Soft-Body and Visuotactile Simulators
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies