Reproduce “APT: Adaptive Personalized Training for Diffusion Models with Limited Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
JungWoo Chae, Jiyoon Kim, JaeWoong Choi, Kyungyul Kim, Sangheum Hwang
LG CNS AI Research and Seoul National University of Science and Technology developed Adaptive Personalized Training (APT), a framework for diffusion models that personalizes generation with limited data. APT mitigates overfitting and preserves prior knowledge by dynamically adjusting training based on an adaptive overfitting indicator and regularizing intermediate feature maps and attention distributions, achieving a 56.1% user preference for APT-generated images, improved text-image similarity, and better preservation of scene context compared to other methods.
Key insights
- Overfitting tendencies in diffusion models are dynamic across different denoising time steps, necessitating an adaptive approach to regularization.
- Stabilizing the mean and variance of intermediate feature representations prevents drastic shifts in the model's internal processing and helps retain generalization.
- Explicitly aligning cross-attention maps is effective for preserving the pre-trained model's semantic understanding and prior knowledge of contextual relationships.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.