Reproduce “ACTS: Learning Visuo-Tactile World Models from Human Physical Interaction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuxiang Ma, Weihan Xu, Pengfei Ye, Anna Mei, Yuxin Ray Song, Tingle Li, Haonan Chen, Edward H. Adelson
ACTS predicts tactile and visual observations from actions combining sensor motion with estimated normal-force change. The authors train it on human interactions and test held-out human and robot teleoperation data. On human PushT scene-view prediction, ACTS reaches 29.69 dB PSNR, versus 27.69 dB for a baseline. In inspected robot rollouts, contact transitions transfer qualitatively better than camera appearance; robot manipulation is not demonstrated.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.