Reproduce “MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shujun Xia, Haokun Lin, Yichen Wu, Yinan Zhou, Zixuan Li, Zhongwei Wan, Xingrun Xing, Yefeng Zheng and 4 more
Researchers from City University of Hong Kong, Harvard Medical School, and other institutions developed MedREK, a retrieval-based framework for medical large language model editing, alongside MedVersa, a batch-editing benchmark. MedREK improves knowledge update efficacy and locality by employing a shared query-key MLP and an attention-based prompt encoder, showing robust performance in both single and batch medical knowledge editing tasks.
Key insights
- A shared query-key MLP, by processing queries and subject-relation knowledge keys in a unified representation space, greatly improves the precision of knowledge retrieval and alignment in medical domains with high textual similarity.
- An attention-based prompt encoder effectively generates high-quality, knowledge-specific prompts, providing dynamic guidance that leads to more accurate and fine-grained knowledge modifications in LLMs.
- Retrieval-based editing, when engineered to handle representation overlap and facilitate batch updates, offers superior preservation of locality and significantly faster edit times compared to traditional parameter-based editing methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.