Reproduce “Gnomon4D: Learning Where and When to Focus for Geometric Motion-Grounded World-Action Modeling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lingwei Dang, Yan Zhou, Juntong Li, Zhenhao Zhang, Yiteng Chen, Xiaoqiang Liu, Liang An, Hongwen Zhang and 5 more
Gnomon4D helps a robot manipulate objects by predicting future video, robot actions, and GeoDelta together. GeoDelta tracks foreground hand and object points’ image-plane displacement and depth change, while interaction-guided attention directs the action expert to their relative motion. On five real-robot tasks, it achieves 82.5% in-distribution and 78.0% out-of-distribution success, but uses measured depth, camera parameters, and foreground masks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.