Reproduce “DreamX: Video Diffusion Model is a Universal Latent Action Learner” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jisoo Kim, Chaehyun Kim, Junsu Kim, Jiho Park, Seongchan Kim, Eunbeen Hong, Soowon Son, Fu-En Yang and 1 more
DreamX transfers motion from human videos to robot actions without paired human–robot training data. It uses learnable queries to read a pretrained video model’s features and compress motion into a code, which can condition future-video generation or robot action prediction. On LARYBench, it achieves 0.17 average normalized action MSE and 69.67% accuracy; transfer is demonstrated only in evaluated open-loop settings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.