Reproduce “A Continued Pretrained LLM Approach for Automatic Medical Note Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dong Yuan, Eti Rastogi, Gautam Naik, Sree Prasanna Rajagopal, Sagar Goyal, Fen Zhao, Bharath Chintagunta, Jeff Ward
DeepScribe Inc. developed HEAL, a 13B LLaMA2-based language model, that autonomously generates comprehensive medical notes from doctor-patient conversations. The model achieved superior completeness with 4.30 missed entities, compared to 11.20 for human scribes, and matched GPT-4's correctness with 0.85 incorrect entities.
Key insights
- Targeted continuous pretraining and explanation tuning on high-quality, domain-specific data enables a smaller, open-source LLM to achieve and surpass the performance of larger, general-purpose models (like GPT-4) and human experts in complex, specialized tasks like medical note generation.
- Balancing general domain datasets with specialized medical data during pretraining is crucial to retain foundational generative capabilities while instilling deep domain knowledge.
- Proprietary real-world doctor-patient dialogues and EHR data, combined with synthetically generated medical instructions, are essential for training an LLM robust enough to interpret and summarize nuanced clinical conversations accurately.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.