Reproduce “When Two Tracers Disagree: An Investigation of Multimodal Fusion for Clinical PET/CT Segmentation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jack A. Johnson, Bartłomiej W. Papież
An investigation into multimodal fusion for prostate cancer PET/CT segmentation found that evaluated fusion strategies, including early and intermediate fusion architectures, did not consistently improve tracer-specific lesion delineation compared to strong single-tracer nnU-Net baselines. While a combined-mask approach achieved a high Dice score, methods designed for tracer-specific outputs showed performance degradation for at least one tracer.
Key insights
- Multimodal fusion, when aiming for tracer-specific outputs, presents a challenge in effectively integrating complementary yet potentially conflicting biological information from PSMA and FDG tracers.
- Strong, self-configuring single-tracer architectures like nnU-Net provide robust baselines that current general-purpose fusion strategies do not consistently surpass.
- Architectures must carefully balance feature representation during fusion to avoid one dominant tracer (e.g., PSMA due to more lesions) overpowering the learning for another (e.g., FDG).
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.