Reproduce “Lens: Rethinking Training Efficiency for Foundational Text-to-Image Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dong Chen, Fangyun Wei, Ziyu Wan, Dongdong Chen, Jiawei Zhang, Jinjing Zhao, Sirui Zhang, Yang Yue and 13 more
The Microsoft Lens Team developed Lens, a 3.8-billion-parameter Text-to-Image model that achieves performance competitive with larger state-of-the-art models while using approximately 19.3% of their training compute. This system also offers faster inference, robust resolution and aspect ratio generalization, and strong multilingual capabilities despite being trained solely on English data.
Key insights
- Training efficiency in T2I models can be systematically optimized by jointly addressing model compactness, data information density (e.g., via dense captioning), and accelerated convergence.
- Direct evaluation of core components like VAEs and language encoders within the full T2I pipeline, rather than proxy metrics, is crucial for selecting those that significantly improve generation performance and accelerate convergence.
- Post-training refinement using RL with a diverse prompt set and structured rubrics, coupled with efficient distillation, can effectively enhance generation quality and inference speed without extensive re-training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.