Reproduce “Enhancing Pancreatic Cancer Staging with Large Language Models: The Role of Retrieval-Augmented Generation” on OpenResearch

About this paper

Hisashi Johno, Yuki Johno, Akitomo Amakawa, Junichi Sato, Ryota Tozuka, Atsushi Komaba, Hiroaki Watanabe, Hiroki Watanabe and 4 more

University of Yamanashi researchers developed a Retrieval-Augmented Generation (RAG) enhanced Large Language Model (NotebookLM) to stage pancreatic cancer, achieving 70% accuracy compared to 38% for its base LLM while also providing transparent justification through retrieved guidelines.

Key insights

• Retrieval-Augmented Generation (RAG) substantially improves LLM accuracy in complex medical classification tasks, demonstrating a clear advantage over simply providing external information within the prompt.
• RAG-LLMs enhance transparency by explicitly presenting retrieved factual excerpts from guidelines, allowing human verification of the model's reasoning.
• The effectiveness of RAG in cancer staging appears generalizable across different cancer types and their intricate classification guidelines.

Summary by alphaXiv.