Reproduce “Seeing and Imagining as One: Unified Video Generation and 4D Perception for Embodied World Modeling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haitang Feng, Zhuoman Liu, Zhennan Wang, Sheng Wang, Juelin Li, Chengzhi Zhao, Hexiang Wei, Lin Shao
GenSight uses one pretrained video-diffusion model to generate future video and infer masks, depth, and 2D flow from observed video. Its shared Video DiT supports dense outputs in both tasks. On RoboMIND task-OOD, it reports FVD 14.55 versus 73.45 and depth RMSE 4.76 cm versus 13.11 cm for the strongest listed baseline; this does not establish robot control or isolate the effect of shared weights.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.