Reproduce “TRACE: Task-Adaptive Reasoning and Representation Learning for Universal Multimodal Retrieval” on OpenResearch

About this paper

Xiangzhao Hao, Shijie Wang, Tianyu Yang, Tianyue Wang, Haiyun Guo, Jinqiao Wang

The TRACE framework integrates generative reasoning with discriminative representation learning, enabling multimodal retrieval systems to deconstruct complex user intents through an adaptive "reasoning then encoding" paradigm. It achieved a new state-of-the-art average Recall@5 of 58.8% on the M-BEIR benchmark, demonstrating significant performance gains on reasoning-intensive tasks and robust zero-shot generalization to unseen scenarios.

Key insights

• Explicit generative reasoning, specifically through Chain-of-Thought, effectively overcomes the "cognitive bottleneck" in static MLLM encoders by allowing the model to deconstruct complex multimodal intents before generating a final embedding.
• The model can implicitly learn to adapt its reasoning depth based on query complexity, dynamically balancing computational cost and retrieval performance without requiring explicit architectural branching.
• Internalizing the deductive process within a unified framework transforms reasoning into a transferable cognitive skill, enabling robust zero-shot generalization to novel retrieval logic and complex semantic constraints in unseen domains.
• Applying explicit reasoning on the query side enhances understanding of user intent, but applying it on the candidate side catastrophically degrades performance by causing embeddings to overfit to linguistic patterns rather than intrinsic visual features.

Summary by alphaXiv.