Reproduce “CtrlWAM: Controllable World Action Models with Aligned Intent and Foresight” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chensheng Peng, Wenhao Ding, Ran Tian, Zewei Zhou, Jef Packer, Maximilian Igl, Peter Karkus, Yan Wang and 4 more
CtrlWAM trains world action models to align predicted actions with video by executing perturbed commands in a simulator and noising the resulting renders while retaining recorded actions and video as targets. In driving forward dynamics, CTRLWAM-M achieves 0.981 m Cmd ADE versus 1.023 m for the SFT Baseline. Evaluation is offline and does not establish closed-loop safety.
Key insights
- The relationship between modalities during denoising matters: simulator-rendered consequences provide a corrective training signal that ordinary independent noise does not.
- Keeping video more corrupted while action estimates evolve can improve action-related consistency and command following, but may trade off against image reconstruction quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.