Reproduce “Balancing Content Size in RAG-Text2SQL System” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Prakhar Gurawa, Anjali Dharmik
This research investigates the optimal balance of content size and quality in Retrieval-Augmented Generation (RAG) for Text-to-SQL systems, demonstrating that excessive detail can increase hallucinations in the Text2SQL model. The study found that a curated context with basic schema and two example INSERT queries provided the best overall performance for an 8B-parameter LLM, outperforming more verbose textual descriptions.
Key insights
- There is a critical trade-off: what improves RAG retrieval (more detailed documents, especially textual descriptions) does not necessarily improve or can even degrade the performance of the Text2SQL model due to prompt complexity and hallucinations.
- Concise, illustrative examples (like INSERT queries) provided sufficient context for the Text2SQL model without causing prompt overload, proving more effective than verbose textual descriptions for combined system performance.
- Column selection, particularly the unnecessary inclusion of ID columns, was identified as a primary source of hallucinations in the generated SQL queries.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.