Reproduce “GIMO: Gaze-Informed Human Motion Prediction in Context” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yang Zheng, Yanchao Yang, Kaichun Mo, Jiaman Li, Tao Yu, Yebin Liu, C. Karen Liu, Leonidas J. Guibas
A new large-scale dataset, GIMO, combined with a dedicated architecture, demonstrates that integrating ego-centric eye gaze information enhances the accuracy of long-term human motion prediction by disambiguating intentions in diverse real-world scenes. The approach reduces prediction errors across various metrics compared to motion-only methods.
Key insights
- Eye gaze serves as a crucial early indicator of human intention, enabling more accurate and unambiguous long-term motion prediction, especially when initial body movements are similar.
- A multi-modal deep learning architecture, particularly utilizing bidirectional cross-modal transformers, effectively fuses gaze, motion, and scene context to leverage gaze information fully.
- High-quality, synchronized multi-modal datasets are essential for training models that can effectively infer and utilize human intention from eye gaze for complex tasks like motion prediction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.