Reproduce “FloMo: 3D Scene Flow as a World Action Model Intermediate for Learning from Human Video” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jeremy Collins, Namra Patel, Ayush Agarwal, Shitij Govil, Animesh Garg
FloMo predicts robot actions and cumulative 3D scene motion from images and instructions, rather than future pixels. It renders motion as RGB and encodes it with a frozen video-model encoder. Across three real-robot tasks, it averaged 91% success versus 51% and 40% for video-predicting variants; on unseen Pull String, it succeeded in 7 of 10 trials, within the tested settings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.