SoMA: A Real-to-Sim Neural Simulator for Robotic Soft-Body Manipulation
Mu Huang, Hui Wang, Kerui Ren, Linning Xu, Mulin Yu, Yunsong Zhou, Bo Dai, Jiangmiao Pang
Abstract
Simulating deformable objects under rich interactions remains a fundamental challenge for real-to-sim robot manipulation, with dynamics jointly driven by environmental effects and robot actions. Existing simulators rely on predefined physics or data-driven dynamics without robot-conditioned control, limiting accuracy, stability, and generalization. This paper presents \textbf{SoMA}, a 3D Gaussian Splat simulator for soft-body manipulation. SoMA couples deformable dynamics, environmental forces, and robot joint actions in a unified latent neural space for end-to-end real-to-sim simulation. Modeling interactions over learned Gaussian splats enables controllable, stable long-horizon manipulation and generalization beyond observed trajectories without predefined physical models. SoMA improves resimulation accuracy and generalization on real-world robot manipulation by 20\%, enabling stable simulation of complex tasks such as long-horizon cloth folding.
BibTeX
@inproceedings{
huang2026soma,
title={So{MA}: A Real-to-Sim Neural Simulator for Robotic Soft-Body Manipulation},
author={Mu Huang and Hui Wang and Kerui Ren and Linning Xu and Yunsong Zhou and Mulin Yu and Bo Dai and Jiangmiao Pang},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=USAkWviwng}
}