Reproduce “M3DR: Towards Universal Multilingual Multimodal Document Retrieval” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Adithya S Kolavi, Vyoman Jain
M3DR, developed by CognitiveLab, presents a framework for universal multilingual multimodal document retrieval across 22 diverse languages, achieving a 152% relative improvement in cross-lingual retrieval (NDCG@5) over baselines and maintaining strong English performance through synthetic data generation and efficient VLM-based models.
Key insights
- A simpler 'Positive-Only' training strategy with in-batch negatives significantly outperformed complex hard negative mining in the multilingual multimodal setting due to sufficient batch diversity.
- Last token pooling proved dramatically superior to mean pooling for decoder-based Vision-Language Models like Gemma 3, becoming the default pooling strategy.
- Matryoshka representation learning successfully enables post-deployment accuracy-efficiency trade-offs, reducing storage requirements by 70% for 768-dimensional embeddings with only a 5% performance drop.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.