Reproduce “Multi-Modal One-Shot Federated Ensemble Learning for Medical Data with Vision Large Language Model” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Naibo Wang, Yuchen Deng, Shichen Fan, Jianwei Yin, See-Kiong Ng

Researchers from the National University of Singapore and collaborators developed FedMME, a one-shot federated learning framework that enhances medical diagnostics by integrating visual features with textual descriptions generated by Vision Large Language Models (VLLMs). This framework consistently outperformed existing one-shot federated learning methods across various medical datasets, demonstrating improved accuracy and robustness while preserving patient privacy and minimizing communication overhead.

Key insights

  • Integrating VLLM-generated textual reports with visual features enhances diagnostic accuracy in medical image analysis within a federated learning setting.
  • Llama-3.2-11B-Vision-Instruct generally yielded more effective textual reports for medical image analysis compared to ChatGPT-4o-Vision.
  • Optimal dimensionality reduction for textual features varies by dataset, indicating the importance of tuning for specific medical applications.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.