Reproduce “LLM-driven Medical Report Generation via Communication-efficient Heterogeneous Federated Learning” on OpenResearch

About this paper

Haoxuan Che, Haibo Jin, Zhengrui Guo, Yi Lin, Cheng Jin, Hao Chen

Researchers at the Hong Kong University of Science and Technology (HKUST) developed FedMRG, a federated learning framework that enables privacy-preserving, multi-center training of Large Language Models for medical report generation. This framework achieved superior performance on intra-domain and unseen domain tests, demonstrating robust generalization and high clinical efficacy by efficiently addressing communication overhead and multi-modal data heterogeneity.

Key insights

• Parameter-Efficient Fine-Tuning (PEFT) methods, specifically LoRA, are crucial for making federated learning feasible with large language models by substantially reducing communication costs.
• Effectively addressing multi-modal data heterogeneity in federated medical report generation requires specialized components for both visual and textual modalities, leveraging both global commonalities and local distinctions.
• Integrating explicit clinical guidance, such as diagnosis-aware prompting, directly into the model's training significantly enhances clinical accuracy and consistency, mitigating the risk of factual errors in generated medical reports.

Summary by alphaXiv.