Reproduce “G3Ego: Gaze-Guided Graphs for Egocentric Action Understanding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Marko Haralović, Akash Ramakrishnan, Estefania Talavera Martinez
G3Ego presents a gaze-guided graph framework for egocentric action understanding that leverages sparsely sampled frames and integrates gaze as a structural cue for graph pruning. This approach achieved the highest Macro-F1 scores on the MECCANO dataset (21.34% for action recognition, 4.20% for action anticipation) and a mean accuracy of 56.49% on EGTEA Gaze+, while significantly reducing graph complexity by 72.4% in nodes and 66.1% in edges compared to full graphs.
Key insights
- Gaze effectively serves as a structural pruning mechanism for action graphs, creating more compact and efficient representations by focusing on action-relevant entities.
- Competitive egocentric action understanding can be achieved using sparsely sampled frames and structured graph modeling, reducing reliance on computationally intensive dense video pretraining.
- The combination of graph-based representations with multimodal input (RGB, hands, VLM captions) improves robustness, particularly for less frequent actions, as reflected by higher Macro-F1 scores.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.