Reproduce “LCM-LoRA: A Universal Stable-Diffusion Acceleration Module” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Simian Luo, Yiqin Tan, Suraj Patil, Daniel Gu, Patrick von Platen, Apolinário Passos, Longbo Huang, Jian Li and 1 more
LCM-LoRA introduces a universal acceleration module for Stable Diffusion models by combining Latent Consistency Distillation with Low-Rank Adaptation and parameter arithmetic. This approach enables high-quality image generation in 2-4 sampling steps across diverse pre-trained and fine-tuned models, achieving substantial speed improvements without additional training.
Key insights
- Latent Consistency Distillation can be made highly parameter-efficient by integrating LoRA, allowing large models to be distilled with reduced computational resources.
- Acceleration capabilities for diffusion models can be modularized into a LoRA "acceleration vector" that is combinable via simple parameter arithmetic.
- This modular approach enables the creation of a "plug-in neural PF-ODE solver" that universally accelerates any fine-tuned Stable Diffusion model or LoRA without further training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.