Reproduce “Hierarchical MoE for Multi-Modal ILD Diagnosis” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Alec K. Peltekian, Gorkem Durak, Halil Ertugrul Aktas, Carrie Lynn Richardson, Mary Carns, Kathleen Aren, GR Scott Budinger, Anthony J. Esposito and 4 more

For Interstitial Lung Disease diagnosis, a hierarchical multimodal Mixture-of-Experts framework from Northwestern University integrates CT imaging and structured EHR data. This system employs adaptive gating to dynamically weigh contributions from different modalities and clinically defined feature groups, demonstrating an AUC of 0.8750 in a retrospective cohort.

Key insights

  • Dynamically weighting the contributions of different data modalities and clinically defined EHR feature groups per patient enhances diagnostic performance and interpretability.
  • Incorporating anatomical priors for imaging and structured clinical groupings for EHR data through a hierarchical expert design promotes specialization and informed routing.
  • Freezing the independently trained imaging expert during multimodal training allows the EHR branch and gating networks to optimize effectively for inter-modality fusion.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.