Reproduce “FreqCa: Accelerating Diffusion Models via Frequency-Aware Caching” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiacheng Liu, Peiliang Cai, Qinming Zhou, Yuqi Lin, Deyang Kong, Benhao Huang, Yupei Pan, Haowen Xu and 4 more
FreqCa introduces a frequency-aware caching framework that accelerates Diffusion Transformers by 6-7x while maintaining generation quality within 2% degradation. It achieves a groundbreaking O(1) memory complexity, reducing memory overhead to approximately 0.07GB, by leveraging a novel Cumulative Residual Feature caching mechanism.
Key insights
- Low-frequency components of DiT features exhibit high similarity but poor continuity across timesteps, whereas high-frequency components show low similarity but high continuity.
- Caching a single Cumulative Residual Feature (CRF) effectively compresses the intermediate states of a Diffusion Transformer, enabling significant memory reduction without substantial fidelity loss.
- Unifying 'cache-then-reuse' for low frequencies and 'cache-then-forecast' for high frequencies optimizes acceleration by adapting to the intrinsic temporal dynamics of different signal components.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.