Reproduce “Contextualizing Biological Language Models across Modalities via Logit-Space Contrastive Alignment” on OpenResearch

About this paper

Yanjun Shao, Yundi Chen, Yashvi Patel, Aurelien Pelissier, María Rodríguez Martínez

LOGICA (Logit-space Contrastive Alignment) enables biological language models to make context-conditioned predictions by performing contrastive learning directly in their output logit space, preserving token-level probabilistic semantics. This framework achieves improved mutation-local variant ranking and provides interpretable cross-modal dependency maps across tasks like protein-ligand binding and TCR-peptide recognition.

Key insights

• Aligning modalities directly in the logit space allows for context-conditioned predictions while retaining the BioFM's native token-level probabilistic interface, which is crucial for mutation-local variant ranking where wild-type terms can exactly cancel.
• The contextualized token probabilities generated by LOGICA can be directly used for three distinct applications: ranking variant candidates, generating interpretable cross-modal dependency maps, and enabling conditional sequence generation.
• The modular design with cross-modal adapters allows various pretrained biological foundation models (e.g., for proteins, immune receptors, small molecules) to be contextualized by different external modalities.

Summary by alphaXiv.