Reproduce “SWAM: Scaling World Action Models with Cross-Modal Alignment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yu Zhou, Xinyuan Xia, Xuankun Yang, Yinghao Yang, Wendi Chen, Bowen Liu, Jianhan Ma, Weicheng Duan and 3 more
SWAM jointly predicts future video and robot actions conditioned on language, retaining video prediction while learning actions. Its video and action branches share masked attention. Across cup stacking, towel folding, and gift-in-box under real-world Hard conditions, SWAM's equally weighted mean complete-task success rate is 40.00%, versus 23.33% for OpenWAM; the result is limited to these three tasks and conditions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.