Reproduce “ConceptFormer: Learning Adaptive Latent Concepts for Query-Document Alignment in Visual Document Retrieval” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chunyi Peng, Zhipeng Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Sen Mei, Yubo Sun, Yongheng Zhang and 4 more
ConceptFormer introduces an adaptive latent concept learning framework for visual document retrieval, bridging the visual-language modality gap by modeling query-relevant evidence as continuous, query-conditioned latent concepts. This approach dynamically allocates representational capacity to focus on fine-grained evidence, leading to improved alignment and retrieval performance across various benchmarks.
Key insights
- Learning continuous, query-conditioned latent concepts effectively bridges the visual-language modality gap in visual document retrieval, outperforming methods based on fixed-granularity or simpler evidence proxies.
- Dynamically adapting the latent concept capacity based on the detected visual evidence significantly improves performance by efficiently representing varying levels of evidence complexity.
- Aligning the ranking distributions induced by the original query and the learned latent concepts is critical for propagating fine-grained, localized relevance signals to the document retriever.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.