Reproduce “UniWAM: Unified World-Action Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wenxuan Song, Jiayi Chen, Jingbo Wang, Shuai Zhou, Xicheng Gong, Zehua Fan, Ziyang Zhou, Junwu E and 8 more
UniWAM combines language-based physical reasoning, future visual prediction, and action generation in one robot model. It trains these capabilities with question-answer data, human videos, and robot demonstrations, and initializes action generation from recent actions. On RoboTwin 2.0, it reaches 68.32% success in clean-to-random evaluation, highest among listed methods, but trails OpenWAM-α in clean-to-clean evaluation.
Key insights
- Human video is not uniformly helpful: it can reduce performance at smaller robot-data scales, but becomes beneficial at roughly 1,000 hours and above in the reported experiments.
- Combining data sources yields incremental gains: average performance rises from 67.53% with robot-only pretraining to 71.34% with human data and 71.73% with VQA added.
- Initializing action generation from recently executed actions improves temporal continuity and, in the reported ablations, outperforms generation from Gaussian noise with fewer denoising steps.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.