Reproduce “CIIR@LiveRAG 2025: Optimizing Multi-Agent Retrieval Augmented Generation through Self-Training” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Alireza Salemi, Mukta Maddipatla, Hamed Zamani
Researchers at the University of Massachusetts Amherst developed mRAG, a multi-agent Retrieval-Augmented Generation framework, which uses self-training with reward-guided trajectory sampling to optimize collaboration among specialized AI agents. The framework demonstrated improved correctness and faithfulness on a DataMorgana-derived test set and ranked 7th in the SIGIR 2025 LiveRAG competition.
Key insights
- A modular multi-agent architecture with specialized agents offers improved scalability and task alignment over single-agent RAG systems.
- A self-training approach, guided by a reward model, effectively optimizes complex inter-agent collaboration, leading to enhanced response quality.
- Evaluation metrics focused purely on recall, like nugget-based scores, can inadvertently incentivize verbosity and may not fully align with desired response quality, suggesting areas for metric refinement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.