Reproduce “Cross-Modal Knowledge Distillation from Spatial Transcriptomics to Histology” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Arbel Hizmi, Artemii Bakulin, Shai Bagon, Nir Yosef
A cross-modal knowledge distillation framework transfers molecularly defined tissue niche structures from spatial transcriptomics (ST) data to histology images. The resulting histology-only model accurately reproduces ST-derived niche organization and cell-type compositions, outperforming morphology-based methods and demonstrating improved correlation with clinical pathology annotations.
Key insights
- Knowledge distillation can effectively transfer complex, neighborhood-level tissue organization patterns (spatial niches) from high-dimensional molecular data to readily available morphological imaging data.
- Integrating molecular guidance from spatial transcriptomics allows histology-based models to identify more biologically meaningful and finer-grained tissue niches than can be achieved by analyzing morphology alone.
- The use of pre-trained backbones (scVI for gene expression, UNIv2 for H&E) and a shared neighborhood-transformer architecture facilitates robust cross-modal information transfer for tissue context learning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.