Reproduce “IN2OUT: Fine-Tuning Video Inpainting Model for Video Outpainting Using Hierarchical Discriminator” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sangwoo Youn, Minji Lee, Nokap Tony Park, Yeonggyoo Jeon, Taeyoung Na
The IN2OUT framework fine-tunes pre-trained video inpainting models for video outpainting by employing a hierarchical discriminator and a specialized outpainting loss function. This approach enhances visual appeal and temporal consistency, yielding PSNR improvements of up to 2.4 dB over baseline inpainting models and processing frames over 52 times faster than some diffusion-based methods.
Key insights
- The primary limitation of applying video inpainting models to outpainting stems from the discriminator's design in adversarial training, rather than the generator's inherent capabilities.
- A single, hierarchically structured discriminator can robustly achieve both local perceptual realism and global consistency without the instability associated with multiple, distinct discriminators.
- Careful formulation of the outpainting loss, by focusing local assessment specifically on the generated regions, is crucial for stable and high-quality fine-tuning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.