ClothCode: Generative Modeling for Garment State Estimation and Dynamics Prediction
cs.RO, cs.CV
Submitted: 2026-10-09
Updated: 2026-10-09
License: http://creativecommons.org/licenses/by/4.0/
The gist: We introduce ClothCode, a unified generative framework for garment state estimation and action-conditioned dynamics prediction in a shared discrete configuration space.
Terminology
Abstract
We introduce ClothCode, a unified generative framework for garment state estimation and action-conditioned dynamics prediction in a shared discrete configuration space. A material-aligned tokenizer encodes garment configuration as residual vector-quantized patch tokens, while task-specific Masked Generative Transformers predict these tokens through parallel confidence-guided refinement. Across three garment templates, ClothCode reduces mean displacement error by 26.6% for state estimation and 37.0% for dynamics prediction relative to the strongest baselines. Dynamics prediction requires only 16.7 ms per window, compared with 556.2 ms for diffusion. Integrated into the same model-predictive controller, ClothCode improves real-world folding and flattening success from 56.7% to 78.9%. Further results and videos are available at: https://anonymous.4open.science/r/ClothCode-DE32
Sources
- Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulation
- AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation
- Diffusion Dynamics Models with Generative State Estimation for Cloth Manipulation
- Learning Mesh-Based Simulation with Graph Networks
- Particle-Grid Neural Dynamics for Learning Deformable Object Models from RGB-D Videos
- ClothTransformer: Unified Latent-Space Transformers for Scalable Cloth Simulation
- Muse: Text-To-Image Generation via Masked Generative Transformers
- Semantic State Estimation in Cloth Manipulation Tasks
- Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies