Reproduce “RAG-NIMBF: A Non-Invasive, Modular Benchmarking Framework for Automated RAG Configuration Analysis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Niclas Hart, Nicolas Weeger
RAG-NIMBF provides a framework for systematically testing retrieval-augmented generation configurations while wrapping existing pipeline components. It varies choices such as chunking, retrieval, and prompts, then evaluates and tracks results. In a preliminary study of a 100-question SQuAD sample, MMR improved retrieval ranking, while faithfulness and prompt effects differed by generator. These findings cover only a subset of configurations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.