Reproduce “AMEGO: Active Memory from long EGOcentric videos” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Gabriele Goletto, Tushar Nagarajan, Giuseppe Averta, Dima Damen
AMEGO, developed by researchers from Politecnico di Torino, FAIR at Meta, and the University of Bristol, introduces an online pipeline that builds a structured, semantic-free memory from very long egocentric videos, inspired by human episodic memory. This system allows for efficient and detailed querying of hand-object interactions and activity-centric locations, achieving a 36.3% accuracy on the new Active Memories Benchmark, surpassing baselines by 12.7 percentage points.
Key insights
- Building an explicit, structured representation that directly captures human activities (hand-object interactions, location visits) rather than relying on uniform video sampling leads to more meaningful and queryable video understanding.
- A 'semantic-free' approach, relying on visual features for object and location distinction rather than predefined labels, enhances flexibility and robustness, allowing fine-grained understanding without external semantic biases.
- The introduction of the Active Memories Benchmark (AMB), featuring over 20,000 language-free visual queries across sequencing, concurrency, and temporal grounding, is crucial for evaluating comprehensive, fine-grained egocentric video understanding.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.