Reproduce “SLA Management in Reconfigurable Multi-Agent RAG: A Systems Approach to Question Answering” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Michael Iannelli, Sneha Kuchipudi, Vera Dvorak

Yext, Inc. researchers introduced a system-theoretic framework for dynamically reconfigurable multi-agent Retrieval Augmented Generation (RAG) systems to manage Service Level Agreements (SLAs) in question answering. The framework integrates non-functional requirements, showing that increasing agents from 3 to 5 improved recall from 0.655 to 0.684, and that aggressive context preprocessing yielded the highest single-agent precision (0.670) and recall (0.656).

Key insights

  • Optimizing context for LLMs is delicate; aggressive pruning of less relevant information can improve answer quality more than sophisticated re-ranking.
  • While increasing the number of RAG agents generally improves performance, this comes with a direct trade-off in increased computational cost and latency.
  • Simple majority voting arbitration mechanisms can achieve comparable performance to more complex relevance-based arbitration, suggesting potential for efficiency gains in specific scenarios.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.