Reproduce “MA4DIV: Multi-Agent Reinforcement Learning for Search Result Diversification” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yiqun Chen, Jiaxin Mao, Yi Zhang, Dehong Ma, Long Xia, Jun Fan, Daiting Shi, Zhicong Cheng and 2 more
MA4DIV introduces a multi-agent reinforcement learning framework that models each document as a cooperative agent to optimize search result diversification. This approach directly optimizes non-differentiable diversity metrics, achieving state-of-the-art performance on large-scale datasets while significantly improving training and inference efficiency.
Key insights
- Search result diversification can be effectively framed as a cooperative multi-agent reinforcement learning problem, allowing for the direct optimization of complex, non-differentiable diversity metrics.
- Modeling the diversification process as a 'one-step' episode, where all document-agents simultaneously choose ranking scores, dramatically enhances training and inference efficiency compared to sequential approaches.
- Value decomposition and off-policy training with a replay buffer enable agents to learn cooperative strategies effectively for a shared global reward, improving both convergence speed and data utilization.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.