Reproduce “ALADIN: Distilling Fine-grained Alignment Scores for Efficient Image-Text Matching and Retrieval” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nicola Messina, Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Fabrizio Falchi, Giuseppe Amato, Rita Cucchiara
ALADIN introduces a two-stage architecture that distills fine-grained alignment scores into an efficient common embedding space, enabling high-performance image-text matching and retrieval. The model achieves competitive recall while demonstrating up to a 90-fold increase in inference speed compared to entangled Vision-Language Transformers.
Key insights
- A disentangled Vision-Language backbone, combined with a fine-grained alignment head, can achieve effectiveness comparable to entangled VL Transformers while offering substantial inference speedups.
- Knowledge distillation, particularly framed as a learning-to-rank problem from fine-grained alignment scores, is highly effective for training an efficient common embedding space.
- Joint training of both alignment and distillation objectives can lead to a slight improvement and regularization of the teacher (alignment head) scores.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.