Reproduce “RAG-based Architectures for Drug Side Effect Retrieval in LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shad Nygren, Pinar Avci, Andre Daniels, Reza Rassol, Afshin Beheshti, Diego Galeano
Researchers developed Retrieval Augmented Generation (RAG) and GraphRAG architectures, integrating a Llama 3-8B model with the SIDER 4.1 database to enhance drug side effect retrieval accuracy. The GraphRAG framework achieved an accuracy of 0.9999, demonstrating a substantial improvement over standalone LLMs, which exhibited accuracies as low as 0.529.
Key insights
- Augmenting LLMs with external, structured knowledge via RAG and GraphRAG frameworks is essential for achieving high accuracy in specialized medical tasks like drug side effect retrieval.
- The structured representation of external knowledge critically impacts performance, with graph-based structures proving superior for capturing complex, relational data compared to text-based formats.
- Standalone LLMs, even large models like ChatGPT, perform poorly in specialized domain-specific tasks without explicit knowledge grounding, making augmentation crucial for reliable medical applications.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.