Reproduce “Geometry-Aware Implicit Memory for Video World Models” on OpenResearch

About this paper

Zhengxuan Wei, Xu Guo, Xinghui Li, Xunzhi Xiang, Min Wei, Yiran Zhu, Qiulin Wang, Xintao Wang and 3 more

GIM-World compresses video history into fixed-size learned tokens and trains them to answer camera-conditioned geometry queries using features from a frozen 3D foundation model. A pruning step removes redundant history frames before encoding. On first-person MIND, it scores 81.70 on normalized reprojection, versus 73.97 for FramePack; this benchmark result does not establish explicit 3D reconstruction.

Key insights

• Directly constraining the implicit memory to encode cross-view scene geometry via camera-queryable supervision significantly improves long-horizon geometric consistency compared to applying geometry signals to the generator itself.
• An information-guided pruning strategy that maximizes mutual information between retained and discarded history frames is crucial for efficiently selecting informative historical context, outperforming uniform or camera-based sampling.
• A compact self-attention mechanism within the memory encoder can drastically reduce computational overhead (less than 0.3% of backbone time) without significant degradation in generation quality, ensuring an efficient memory module.

Summary by alphaXiv.