Reproduce “Understanding Human Gaze Communication by Spatio-Temporal Graph Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lifeng Fan, Wenguan Wang, Siyuan Huang, Xinyu Tang, Song-Chun Zhu
Researchers from UCLA VCLA and collaborators introduced a novel problem of understanding human gaze communication in social videos, formalizing it into atomic and event levels. They contributed VACATION, a large-scale, richly annotated dataset, and developed a spatio-temporal graph reasoning model that achieves 55.02% top-1 accuracy for atomic-level gaze communication and 37.1% for event-level classification on this new dataset.
Key insights
- Explicitly learning the social graph structure with ground-truth supervision is crucial for accurate atomic-level gaze communication understanding, as shown by a significant performance drop when learned implicitly.
- Temporal reasoning, implemented via an LSTM, contributes to improved performance in atomic-level gaze classification, indicating the importance of modeling dynamics over time.
- The quality of event-level gaze communication prediction is heavily dependent on the accuracy of the underlying atomic-level classifications.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.