Reproduce “CineMobile: On-Device Image-to-Video Diffusion for Cinematic Camera Motion Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xuyao Huang, Zelai Deng, Xu Wang, Xizhong Xiao, Zhijie Deng
CineMobile introduces an on-device image-to-video diffusion framework designed for generating cinematic camera motion effects like bullet time and dolly zoom. The optimized Diffusion Transformer (DiT) model achieves a 40x speedup in denoising time and reduces FLOPs by 71.89x on mobile platforms while maintaining visual quality and temporal consistency.
Key insights
- Structured depth pruning, which maintains the hidden dimension of the backbone, effectively reduces video DiT complexity without degrading temporal modeling or fine-grained details.
- A combination of supervised warm-up and adversarial distillation, integrating Generalized Reinforcement Policy Optimization (GRPO), successfully enables stable and high-quality few-step generation from a pruned DiT.
- Focusing on controllable cinematic camera motion effects provides a more robust target for aggressive model acceleration, helping to maintain visual and semantic integrity in on-device I2V generation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.