Reproduce “NoisEasier: Test-Time Noise Optimization for Text-to-Video Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yujiang Pu, Yu Kong
NoisEasier presents a test-time noise optimization framework for text-to-video generation that enhances fine-grained compositional alignment using differentiable rewards and no model retraining. By leveraging Video Consistency Models for efficiency and a multi-objective, negative-aware reward function, the framework improved compositional metrics on benchmarks, demonstrating up to a 12.66% gain in spatial relation and a 22.12% gain in multiple objects on T2V-Turbo.
Key insights
- Jointly optimizing both initial latent noise and intermediate stochastic perturbations throughout the denoising trajectory provides finer control and faster convergence for compositional alignment compared to optimizing only the initial latent.
- Utilizing Video Consistency Models (VCMs) is essential for overcoming the computational bottleneck, making end-to-end gradient propagation for test-time noise optimization practical in T2V generation.
- Negative-aware reward calibration, implemented with semantically contrasting prompts generated by an LLM, significantly sharpens the sensitivity of reward models to fine-grained compositional errors, improving semantic disentanglement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.