Reproduce “RoFormer for Position Aware Multiple Instance Learning in Whole Slide Image Classification” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Etienne Pochet, Rami Maroun, Roger Trullo
RoFormerMIL adds relative spatial reasoning to weakly supervised whole-slide image classification by applying a two-dimensional rotary-position transformer layer to tissue-patch features before MIL pooling. Across TCGA-NSCLC, Camelyon16, and BRACS, the approach improves slide-level AUROC over ABMIL and DSMIL baselines, with the largest gains on Camelyon16 and better performance than absolute positional encoding.
Key insights
- Relative positional encoding is better suited to variable WSI layouts than absolute sine-cosine positions: on TCGA-NSCLC, RoFormer reached 0.92 AUROC versus 0.75 for a ViT-style absolute-position model.
- The gains are not explained only by increasing model size; a roughly 4.3-million-parameter RoFormer model outperformed larger ABMIL variants without spatial attention.
- A single spatial-context layer can improve established MIL architectures while preserving their modular slide-level aggregation design.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.