Reproduce “T-RAG: Lessons from the LLM Trenches” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Masoomali Fatehkia, Ji Kim Lucas, Sanjay Chawla
The Qatar Computing Research Institute (QCRI) developed T-RAG, a system for secure, on-premise question answering over private organizational documents by integrating Retrieval-Augmented Generation (RAG) with a QLoRA-finetuned Llama-2 7B model and a unique tree-based context for hierarchical information. T-RAG achieved a 73.0% total correct rate in human evaluations, surpassing RAG (56.8%) and finetuned-only (54.1%) models, especially for hierarchical queries, and demonstrated improved robustness in "Needle in a Haystack" tests.
Key insights
- Combining finetuning and Retrieval-Augmented Generation (RAG) creates a synergistic approach that improves domain-specific question answering accuracy and resource efficiency.
- Explicitly structuring hierarchical organizational knowledge in a tree-based context dramatically enhances an LLM's ability to answer entity-related questions, reducing hallucinations and misaligned answers.
- Finetuning an LLM with methods like QLoRA on a focused dataset can significantly adapt it to a domain without substantial loss of general language understanding.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.