Physical-Field-Bridged Material Parameter Inference from Multi-View Deformation Videos
cs.CV, cs.RO
Submitted: 2026-10-13
Updated: 2026-10-13
Code: https://github.com/3365538768/DeformFieldBench
License: http://creativecommons.org/licenses/by/4.0/
The gist: Estimating simulation-ready material parameters from visual observations is crucial for deformable-object manipulation but challenging because material properties are only indirectly revealed through
Abstract
Estimating simulation-ready material parameters from visual observations is crucial for deformable-object manipulation but challenging because material properties are only indirectly revealed through physical responses. We introduce physical-field-bridged material inference, a single-pass framework that uses dense physical responses to connect multi-view RGB deformation videos with constitutive-type classification and continuous parameter regression. Unlike inverse-physics methods requiring per-scene reconstruction, online tuning, or simulation-in-the-loop optimization, our model predicts force/contact masks, projected motion-flow fields, and projected stress-response maps, then conditions material inference on these fields to reduce reliance on appearance or category shortcuts. We construct DeformFieldBench, a 3DGS MPM learning environment comprising 5,000 simulations across 100 object shapes and five mechanically informative loading modes, with dense pixel-aligned physical supervision. Our method achieves the best median errors on all four continuous parameters, improves stress-hotspot recall from 0.0762 to 0.5121, and attains 0.8676 SSIM/36.82 dB PSNR in same-simulator replay on unseen objects. Under a protocol-matched GSO evaluation, inference and future-state prediction take approximately 30 seconds, compared with 15-120 minutes for per-scene optimization. The method also yields measurable replay agreement on five real-world sequences. These results support dense physical fields as an effective bridge for fast material inference, while cross-simulator and real-world tests reveal remaining domain and identifiability gaps.
Sources
- MASIV: Toward Material-Agnostic System Identification from Videos
- GIC: Gaussian-Informed Continuum for Physical Property Identification and Simulation
- Learning Physics-Grounded 4D Dynamics with Neural Gaussian Force Fields
- EMPM: Embodied MPM for Modeling and Simulation of Deformable Objects
- PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos
- PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identification
- PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation
- GaussianProperty: Integrating Physical Properties to 3D Gaussians with LMMs
- PUGS: Zero-shot Physical Understanding with Gaussian Splatting
- Physion: Evaluating Physical Prediction from Vision in Humans and Machines
- ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation
- DOFS: A Real-world 3D Deformable Object Dataset with Full Spatial Information for Dynamics Model Learning
- PokeFlex: A Real-World Dataset of Volumetric Deformable Objects for Robotics
- DreamGaussian4D: Generative 4D Gaussian Splatting
- DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation
- PhysX-3D: Physical-Grounded 3D Asset Generation
- PhysWorld: From Real Videos to World Models of Deformable Objects via Physics-Aware Demonstration Synthesis
- UniPhy: Learning a Unified Constitutive Model for Inverse Physics Simulation
- Learning Neural Constitutive Laws From Motion Observations for Generalizable PDE Dynamics
- Pixie: Fast and Generalizable Supervised Learning of 3D Physics from Pixels
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models