Reproduce “Grouped Speculative Decoding for Autoregressive Image Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junhyuk So, Juncheol Shin, Hyunho Kook, Eunhyeok Park
Researchers from POSTECH propose Grouped Speculative Decoding (GSD) to accelerate autoregressive (AR) image generation by modifying the token acceptance criterion to consider semantically plausible token groups. This training-free method achieves up to 4.65x latency reduction and 3.91x NFE acceleration compared to vanilla AR decoding, while maintaining or even slightly improving image quality.
Key insights
- The low acceptance rate of speculative decoding in AR image generation stems from the inherent redundancy and diversity of visual tokens, leading to high-entropy predictions where many tokens are plausible but individually low-probability.
- Grouping semantically similar visual tokens and basing speculative decoding acceptance on these groups significantly improves the acceptance rate by smoothing out subtle probability differences between probability distributions.
- Effective clustering for visual tokens requires a dynamic, context-aware approach, utilizing the expert model's predicted probabilities to identify relevant neighbors, as static embedding-based clustering fails to capture contextual semantic changes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.