Reproduce “Your Embedding Model is SMARTer Than You Think” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jianrui Zhang, Hyun Jung Lee, Sukanta Ganguly, Tae-Eui Kam, Donghyun Kim, Yong Jae Lee
SMART reuses token-level representations from frozen multimodal retrievers to add late-interaction matching to their pooled retrieval scores, without parameter updates in its inference-only form. On MMEB-V2’s 24 visual-document tasks, Qwen3-VL-Embedding-2B improves task-averaged NDCG@5 from 79.27 to 80.10. Gains are strongest for fine-grained retrieval; classification changes little, while token indexing and matching add serving costs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.