Reproduce “Semi-Autoregressive Transformer for Image Captioning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuanen Zhou, Yong Zhang, Zhenzhen Hu, Meng Wang
This paper presents SATIC (Semi-Autoregressive Transformer for Image Captioning), a model that balances inference speed and caption quality by generating word groups autoregressively while generating words within each group in parallel. The approach, enhanced with sequence knowledge distillation, achieves significantly faster decoding than autoregressive models with comparable caption quality on the MSCOCO benchmark.
Key insights
- The semi-autoregressive generation, combining group-wise sequential processing with intra-group parallel generation, provides a flexible and controllable trade-off between inference speed and caption quality.
- Sequence Knowledge Distillation (SeqKD) is crucial for mitigating quality degradation associated with parallel decoding, effectively transferring fluency and accuracy from a strong autoregressive teacher model.
- Weight initialization from a pre-trained autoregressive model significantly aids convergence and improves quality, particularly for larger group sizes, while self-critical training further refines outputs for better metric scores.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.