Reproduce “Devil is in the Queries: Advancing Mask Transformers for Real-world Medical Image Segmentation and Out-of-Distribution Localization” on OpenResearch

About this paper

Mingze Yuan, Yingda Xia, Hexin Dong, Zifan Chen, Jiawen Yao, Mingyan Qiu, Ke Yan, Xiaoli Yin and 8 more

A framework called MaxQuery enhances Mask Transformers for medical image analysis, delivering improved segmentation of known tumor types while robustly localizing out-of-distribution (OOD) tumors at a pixel level. This method achieves an average increase of 7.39% in AUROC and 14.69% in AUPR for OOD localization, along with a 5.27% improvement in Dice Score for inlier segmentation on multi-type tumor CT datasets.

Key insights

• Object queries within Mask Transformers, when properly guided, can effectively serve as learnable cluster centers for inlier conditions, and their affinities to pixel features can be leveraged for reliable pixel-level OOD localization.
• Explicitly enforcing a structured distribution for object queries using a Query-Distribution (QD) loss is crucial for preventing redundant query allocation and forcing queries to focus discriminatively on relevant foreground regions, enhancing both segmentation and OOD performance.
• Leveraging pre-softmax query responses for OOD anomaly scoring is empirically more robust than post-softmax scores or category-level methods, as it better captures a pixel's true dissimilarity from known inlier patterns without being diluted by normalization.

Summary by alphaXiv.