Reproduce “Multi-Person 3D Motion Prediction with Multi-Range Transformers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiashun Wang, Huazhe Xu, Medhini Narasimhan, Xiaolong Wang
MRT forecasts several people’s future 3D motion by separately encoding each person’s movement history and the scene-wide history, then using a single pose from the target person to query both. It recursively predicts joint-position offsets. In constructed Mix1 scenes with 9–15 people, its 3-second MPJPE is 3.97 versus SocialPool’s 4.84, in 0.1-meter units.
Key insights
- Decomposing motion modeling into distinct local-range (individual) and global-range (social interaction) Transformer encoders allows for comprehensive and accurate multi-person 3D motion prediction.
- Querying the Transformer decoder with a single static pose, rather than a full sequence, acts as a bottleneck that encourages the encoders to learn more complex and exploitable representations of motion and interaction.
- The integration of Spatial Positional Encoding (SPE) within the global-range encoder output aids in implicitly clustering individuals into interaction groups, while selective application of DCT/IDCT improves motion representation and prevents freezing.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.