Reproduce “EgoPoints: Advancing Point Tracking for Egocentric Videos” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ahmad Darkhalil, Rhodri Guerrier, Adam W. Harley, Dima Damen
A dedicated benchmark, EgoPoints, was introduced to evaluate dense point tracking in egocentric videos, alongside K-EPIC, a pipeline for generating semi-real training data. Fine-tuning state-of-the-art models with K-EPIC improved their performance on egocentric footage, for example, enhancing CoTracker's ReIDδ_avg from 4.8% to 7.2% while largely preserving general performance.
Key insights
- State-of-the-art point trackers exhibit significant performance degradation (e.g., CoTracker's δ_avg decreased from 74.7% to 38.5%) and very poor re-identification capabilities on egocentric videos.
- A mixed-data fine-tuning strategy using semi-real egocentric data from K-EPIC effectively adapts models, improving egocentric tracking and re-identification performance without causing substantial catastrophic forgetting on general benchmarks.
- Both static scene elements and dynamic objects that undergo out-of-view events and require re-identification are critical for achieving robust egocentric point tracking, with combining both yielding the best improvements.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.