Reproduce “A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Taehong Moon, Moonseok Choi, EungGu Yun, Jongmin Yoon, Gayoung Lee, Jaewoong Cho, Juho Lee
This paper presents Adaptive Score Estimation (ASE), a framework that accelerates diffusion model sampling by adaptively skipping parts of the neural network during inference. ASE reduces computational cost per step by leveraging a time-varying block-dropping schedule, achieving over 30% acceleration with minimal quality degradation on various image synthesis tasks.
Key insights
- The complexity of the score estimation task in diffusion models varies across different timesteps, with earlier noisy steps requiring less computational power than later, more refined steps.
- Early-exiting techniques, commonly used in large language models to save computation, can be adapted to the time-dependent nature of diffusion models.
- A carefully designed fine-tuning process, involving limited updates and loss reweighting, can adapt a diffusion model to a block-dropping schedule without significantly compromising generation quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.