RoboSimStudio: An Interactive Toolkit for Human and Agent Rapid Real2Sim of Robot Manipulation Tasks
cs.RO, cs.AI, cs.GR, cs.LG
Submitted: 2026-10-13
Updated: 2026-10-14
Comments: 13 pages, 7 figures, 1 table. Project website: https://robosimstudio.github.io
Code: https://github.com/newton-physics/newton
Project page: https://robosimstudio.github.io
License: http://creativecommons.org/licenses/by/4.0/
The gist: It has been difficult to accurately model the physics of the interaction between a robot and the objects it manipulates, including contact mechanics, coupled contact dynamics, deformations, stiffness
Abstract
It has been difficult to accurately model the physics of the interaction between a robot and the objects it manipulates, including contact mechanics, coupled contact dynamics, deformations, stiffness and friction. We present RoboSimStudio (RSS), an interactive toolkit that makes choices of solver, contact model, and physical parameters explicit and editable, allowing both humans and coding agents to quickly create and iteratively refine simulations based on observed physical behavior. We implement RSS with the open Newton physics simulation system and describe experiments with human-guided real2sim using (1) visual evaluation, (2) force-feedback and (3) agentic Real2Sim2Real. Code, video and additional information can be found on the project website: https://robosimstudio.github.io
Sources
- Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning
- robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
- Video2Policy: Scaling up Manipulation Tasks in Simulation through Internet Videos
- AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents
- Maestro: Orchestrating Robotics Modules with Vision-Language Models for Zero-Shot Generalist Robots
- ASPIRE: Agentic /Skills Discovery for Robotics
- DexAgent: An Agentic Human2Sim2Robot Framework for Dexterous Manipulation with Self-Evolving Tool Library
- Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents
- cuRobo: Parallelized Collision-Free Minimum-Jerk Robot Motion Generation
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies