Reproduce “Totem: Object-Centric 3D Memory for World-Consistent Video Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zichen Wan, Tianyu Huang, Fan Li, Kai Wang, Hongzhi Zhang, Wangmeng Zuo

Totem maintains selected rigid objects as persistent 3D memories in a shared world for single-image, camera-controlled video generation. It renders object appearance and scene context along target camera paths, while attention links surfaces across views and encodes 3D relations between objects. On a 400-video OOD benchmark, adding object-centric attention improves object-level consistency metrics, but static memories limit suitability for non-rigid scenes.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.