Reproduce “Perceive, Predict, and Plan: a Unified World Action Model for Egocentric Dexterous Manipulation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zirui Wu, Chen Guo, Tianjian Jiang, Zeren Jiang, Zeming Li, Lingchen Yang, Longteng Duan, Ying Xue and 6 more

P³-WAM unifies egocentric video and hand-and-camera motion for perception, prediction, and goal-conditioned planning. The model uses joint attention to let video and motion condition or be predicted together. After 5,000 steps, unified training lowers macro camera-space MPJPE from 113.537 to 86.898 mm and raises PSNR from 17.356 to 18.138 dB. Unequal task exposure means gains support the recipe, not shared representations alone.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.