Reproduce “Chain-of-Thought Re-ranking for Image Retrieval Tasks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shangrong Wu, Yanghong Zhou, Yang Chen, Feng Zhang, P. Y. Mok
The Chain-of-Thought Re-Ranking (CoTRR) framework utilizes Multimodal Large Language Models (MLLMs) for direct, listwise re-ranking in image retrieval, achieving state-of-the-art performance with significant R@1 improvements on CIRR (+12.41%) and Flickr30K (+5.73%) across text-to-image, composed image, and chat-based retrieval tasks.
Key insights
- Directly involving MLLMs in listwise re-ranking, guided by Chain-of-Thought, unlocks their full reasoning potential for complex, globally consistent comparisons among candidate images.
- Structured query deconstruction and detailed, component-wise image evaluation provide fine-grained, interpretable insights into the MLLM's ranking decisions, enhancing transparency.
- The training-free, prompt-based approach allows for broad applicability across diverse retrieval tasks (TIR, CIR, Chat-IR) and various MLLMs without requiring task-specific fine-tuning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.