Reproduce “Protriever: End-to-End Differentiable Protein Homology Search for Fitness Prediction” on OpenResearch

About this paper

Ruben Weitzman, Peter Mørch Groth, Lood Van Niekerk, Aoi Otani, Yarin Gal, Debora Marks, Pascal Notin

Protriever introduces an end-to-end differentiable framework for protein homology search that learns to identify task-relevant sequences for protein fitness prediction. The system achieves state-of-the-art performance among sequence-based models and offers a two orders of magnitude speedup over traditional homology search methods.

Key insights

• End-to-end differentiable training of retriever and reader components allows the retrieval process to learn what constitutes 'useful' homology for a specific downstream task, moving beyond generic sequence similarity.
• Joint training (EMDR loss) with a conditional autoregressive objective for the reader directly guides the retriever to prioritize sequences that improve language model performance.
• Pre-training the retriever with a Dense Passage Retrieval (DPR) paradigm and initializing the reader with a pre-trained PoET model significantly contributes to overall performance, especially for low-depth MSAs.

Summary by alphaXiv.