Reproduce “Householder Motion Embedding: Noise Modulation for Motion-Controllable Video Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tao Jun Lin, Jinhong Ni, Ruyi Zha, Yifu Wang, Xibin Song, Pan Ji, Hongdong Li
HME encodes motion guidance for image-to-video generation by applying flow-dependent orthogonal transformations to noise channels at locations, preserving spatially independent standard-Gaussian noise while introducing motion-dependent relationships across frames. On DAVIS’s 45-video motion-transfer evaluation, HME lowers optical-flow endpoint error from 5.644 to 4.486 and raises CoTracker mIoU from 0.464 to 0.548 versus GWTF, though released checkpoints and unequal training data limit comparisons.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.