Reproduce “EgoGeneralist: Unifying Egocentric Action-Video Tasks through Video-Native Modeling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yanlin Jin, Bowen Yang, Cheng Zhang, Yifan Yang, Zhe Li, Jianfei Cai, Kai Zhu
EgoGeneralist uses a pretrained video model to generate first-person video conditioned on rendered metric action and recover metric action from Posegrid latents. It renders action as an ego-view map for video control and a multiview Posegrid for action readout. On 99 HOT3D target-reaching tasks, it records 3.81 cm mean minimum distance versus VITRA’s 6.32 cm, without establishing real-world robot execution.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.