Reproduce “Large Language Model Distilling Medication Recommendation Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qidong Liu, Xian Wu, Xiangyu Zhao, Yuanshao Zhu, Zijian Zhang, Feng Tian, Yefeng Zheng
LEADER is a two-stage framework that distills the semantic understanding of a fine-tuned Large Language Model into a compact student model for medication recommendation. This approach enhances recommendation accuracy and effectively addresses the cold-start problem for single-visit patients, while significantly reducing inference cost and memory usage, achieving up to 30x acceleration.
Key insights
- Feature-level knowledge distillation is effective in transferring the rich semantic understanding of LLMs to smaller models, outperforming traditional output-level distillation.
- A modified LLM output layer can convert free-form text generation into a multi-label classification task over a predefined drug vocabulary, mitigating the "out-of-corpus" problem.
- Aligning patient profile representations with medication concepts via contrastive learning enables robust medication recommendations for single-visit patients.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.