Reproduce “MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tianhong Li, Huiwen Chang, Shlok Kumar Mishra, Han Zhang, Dina Katabi, Dilip Krishnan
MAGE unifies class-unconditional image generation and self-supervised representation learning within a single masked image modeling framework. The approach achieves state-of-the-art performance in both high-fidelity image synthesis and learning transferable visual features on ImageNet-1K.
Key insights
- Viewing image generation and representation learning as two ends of a masking ratio spectrum allows for their unification within a single masked image modeling framework.
- Semantic tokenization is crucial for high-quality, diverse image generation and for learning robust, transferable representations by enabling the model to operate at a higher semantic level.
- The use of a learned, image-specific class token for padding encoder output, rather than a universal mask token, significantly enhances both generative and representational capabilities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.