Reproduce “Towards Scalable SOAP Note Generation: A Weakly Supervised Multimodal Framework” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Sadia Kamal, Tim Oates, Joy Wan

Researchers from University of Maryland Baltimore County and Johns Hopkins University develop a weakly supervised multimodal framework for generating clinically structured SOAP notes from dermatological images and sparse clinical features, achieving perfect 20/20 scores on LLM-as-a-Judge evaluation and superior ClinicalBERT F1 performance compared to GPT-4o, Claude, and DeepSeek through Vision-LLaMA 3.2 fine-tuning on synthetic training data created via retrieval-augmented generation from authoritative medical sources, addressing the critical bottleneck of scarce expert-annotated datasets in medical AI by using only the PAD-UFES-20 dataset's 2,298 dermoscopic images with structured metadata to generate weakly supervised training targets, with novel clinical evaluation metrics (MedConceptEval and Clinical Coherence Score) demonstrating statistically significant variation across SOAP sections but consistent performance across six dermatological conditions including melanoma, nevus, and basal cell carcinoma.

Key insights

  • Weak supervision through RAG-enhanced synthetic data generation can effectively replace expensive expert annotation while maintaining clinical quality and accuracy
  • Assessment and Plan sections consistently achieve higher semantic alignment with clinical concepts compared to Subjective and Objective sections across all dermatological conditions
  • SOAP section structure has statistically significant impact on semantic quality (F(3,20) = 3.88, p=0.024) while lesion type shows no significant effect, indicating consistent cross-condition performance
  • ClinicalBERT evaluation metrics reveal superior medical domain alignment compared to general-purpose BERT, highlighting importance of domain-specific assessment in medical AI

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.