Reproduce “SSD: Spatial-Semantic Head Decoupling for Efficient Autoregressive Image Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Siyong Jian, Huan Wang
SSD (Spatial-Semantic Head Decoupling) introduces a novel framework for efficiently optimizing the Key-Value (KV) cache in autoregressive image generation models like Janus-Pro. This approach achieves up to a 5x reduction in peak memory usage and up to a 10.7x increase in inference throughput while preserving high visual quality, making high-resolution image generation practical on resource-constrained hardware.
Key insights
- Attention patterns in autoregressive visual generation models exhibit a clear dichotomy between localized spatial processing and global semantic aggregation.
- Semantic information from textual prompts is concentrated non-uniformly, with 'margin columns' of the raster-scanned image grid acting as key semantic anchors.
- Tailored KV cache compression, which differentiates and applies specific strategies to spatial and semantic attention heads, is crucial for efficient visual autoregressive generation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.