Reproduce “EgoCaptioner: Towards Comprehensive and Fine-Grained Egocentric Video Captioning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yuqi Tang, Yang Shi, Yuran Wang, Tengfei Liu, Rundong Wang, Zongyang Qiu, Qixun Wang, Zhuoran Zhang and 9 more

EgoCaptioner pairs video overview with timed, ordered steps whose detail adapts to activity type. It uses coarse-to-fine annotation to establish event structure before expanding step descriptions and trains captions with complementary rewards. In a matched comparison, it improves EgoTempo accuracy from 26.60% to 36.40% and EgoHome PSJ from 62.89 to 71.63; downstream assistant or robot performance is not demonstrated.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.