Reproduce “Once Is Enough: Lightweight DiT-Based Video Virtual Try-On via One-Time Garment Appearance Injection” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yanjie Pan, Qingdong He, Lidong Wang, Bo Peng, Mingmin Chi
The "ONCE IS ENOUGH" (OIE) framework introduces a lightweight Diffusion Transformer (DiT) approach for video virtual try-on, injecting garment appearance only once into the first frame and employing LoRA for efficient fine-tuning. This method achieves leading video quality with significantly reduced computational overhead and parameter count compared to existing DiT-based solutions.
Key insights
- Effective garment appearance injection can be achieved by processing only the first frame and encoding its details into the latent space, leveraging a large DiT's temporal priors for consistent propagation.
- Parameter-efficient fine-tuning techniques, such as LoRA, are crucial for adapting powerful pre-trained DiT models to specialized video generation tasks without incurring high computational and memory costs.
- A modular approach combining a specialized image try-on for initial detail, a mask guider for background preservation, and a LoRA-tuned video DiT efficiently balances high-fidelity generation with resource optimization.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.