Reproduce “WixQA: A Multi-Dataset Benchmark for Enterprise Retrieval-Augmented Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dvir Cohen, Lin Burg, Sviatoslav Pykhnivskyi, Hagit Gur, Stanislav Kovynov, Olga Atzmon, Gilad Barkan
Wix.com AI Research introduces WixQA, a multi-dataset benchmark and enterprise-scale knowledge base designed to evaluate Retrieval-Augmented Generation (RAG) systems in real-world customer support scenarios. It features datasets requiring multi-article synthesis and procedural guidance, establishing baselines that highlight challenges and opportunities for RAG advancement in enterprise AI.
Key insights
- Dense retrieval models (E5) demonstrate clear superiority over keyword-based methods (BM25) for Context Recall, especially in enterprise datasets requiring multi-article information synthesis.
- The WixQA datasets exhibit varying difficulty levels, with WixQA-Simulated proving the most challenging, indicating the complexity of generating maximally accurate and concise procedural guidance from complex underlying information.
- Current RAG systems still have substantial room for improvement in handling complex enterprise-specific queries that demand detailed, multi-step, and multi-source answers.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.