Reproduce “Composed Multi-modal Retrieval: A Survey of Approaches and Applications” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kun Zhang, Jingyu Li, Zhe Li, Jingjing Zhang, Fan Li, Yandong Liu, Rui Yan, Zihang Jiang and 5 more
Researchers at the University of Science and Technology of China deliver a comprehensive survey of Composed Multi-modal Retrieval (CMR), systematically categorizing over 250 works, six types of visual-linguistic combiners, seven loss functions, and detailing seven application domains. The work consolidates existing knowledge and identifies key future research directions in this evolving field.
Key insights
- Composed Multi-modal Retrieval (CMR) represents a paradigm shift, enabling fine-grained, context-aware retrieval by allowing users to combine visual inputs with natural language modifications.
- The field's rapid advancement is heavily driven by the increasing reliance on large pre-trained Vision-Language Models (VLMs) like CLIP, which provide strong foundational representations.
- Overcoming data scarcity is crucial, leading to significant research into zero-shot and semi-supervised learning techniques for automatic pseudo-data generation and noise mitigation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.