Reproduce “One Prior, Two Dynamics: Repurposing Video Diffusion Transformers for Self-Supervised Latent Action Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuxin Jiang, Wenqiang Sun, Jiaqi Li, Yuchao Gu, Lvmin Zhang, Ivor Tsang, Mike Zheng Shou
O²LaF learns compact actions from video transitions and uses them to predict future frames. It adapts a shared pretrained video generator for inverse and forward dynamics, with causal attention limiting access to future frames. On a few-shot robot-probe benchmark, its frozen action features reach wrist-action EEF R² of 0.277, versus 0.149 for DreamDojo. This supports motion transfer to probes, not direct robot control.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.