Reproduce “Multimodal Structure Preservation Learning” on OpenResearch

About this paper

Chang Liu, Jieshi Chen, Lee H. Harrison, Artur Dubrawski

A framework called Multimodal Structure Preservation Learning (MSPL) learns representations for one data type (e.g., MALDI-TOF spectra) by explicitly preserving the clustering structure from a more informative source (e.g., Whole Genome Sequencing SNP distances). This method enables cheaper, faster modalities to achieve the discriminatory power needed for tasks like microbial outbreak detection, reducing costs in public health surveillance.

Key insights

• MSPL introduces "structure-level alignment" by explicitly transferring clustering structure from a high-value modality to a more accessible one, deviating from traditional feature-level alignment.
• The explicit dissimilarity matching objective in MSPL is crucial for preserving external structure, outperforming baselines that rely on implicit structure recovery or direct cluster label classification, especially in sparse data scenarios.
• MSPL demonstrates robustness across diverse data substructures (e.g., species with high Shannon entropy) and is largely agnostic to the difficulty of its pretext task, indicating broad applicability.

Summary by alphaXiv.