Reproduce “DreamLite: A Lightweight On-Device Unified Model for Image Generation and Editing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kailai Feng, Yuxiang Wei, Bo Chen, Yang Pan, Hu Ye, Songwei Liu, Chenqian Yan, Yuan Gao
Researchers at ByteDance developed DreamLite, a 0.39B parameter diffusion model that unifies text-to-image generation and text-guided image editing on mobile devices. The model achieves 1024x1024 image processing in under one second on a flagship smartphone, demonstrating competitive performance against larger server-side models while operating entirely on-device.
Key insights
- The in-context conditioning paradigm, guided by task tokens, successfully unifies distinct generative (T2I) and editing tasks within a single compact diffusion model.
- A task-progressive pretraining scheme, particularly with foreground-emphasis masking during editing pretraining, is crucial for stable convergence and optimal performance in compact, multi-task diffusion models.
- Combining a compact architecture with supervised fine-tuning, reinforcement learning, and step distillation enables real-time on-device performance while maintaining high visual quality and instruction following.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.