Reproduce “Benefits of Linear Conditioning with Metadata for Image Segmentation” on OpenResearch

About this paper

Andreanne Lemay, Charley Gros, Olivier Vincent, Yaou Liu, Joseph Paul Cohen, Julien Cohen-Adad

This research from Polytechnique Montreal and Mila integrates external metadata into deep learning models for medical image segmentation using a linear conditioning technique called Feature-wise Linear Modulation (FiLM). The approach demonstrates improved segmentation performance for spinal cord tumors, achieves robust learning with missing labels in multi-organ segmentation, and enables efficient adaptation to tasks with limited data.

Key insights

• Explicitly conditioning U-Net models with relevant discrete metadata, such as tumor type, significantly enhances segmentation accuracy by providing valuable contextual cues not present in image pixels alone.
• FiLM enables robust training of multi-class segmentation models even when annotations are incomplete or missing for certain classes, by guiding the network towards the specific target organ or structure.
• A single FiLMed model can efficiently adapt to multiple segmentation tasks or scenarios with small and unbalanced datasets, outperforming traditional U-Nets by leveraging information across more abundant tasks.

Summary by alphaXiv.