Reproduce “Structure-Aligned Protein Language Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Can Chen, David Heurtel-Depeiges, Robert M. Vernon, Christopher James Langmead, Yoshua Bengio, Quentin Fournier
A post-training framework enhances sequence-only protein language models by integrating both inter-protein and intra-protein structural knowledge. This method improves performance on contact prediction and zero-shot deep mutational scanning, exhibiting a deeper biophysical understanding of proteins without requiring structural input during inference.
Key insights
- Integrating both inter-protein (global via pGNNs) and intra-protein (local via structure tokens) structural information through a dual-task framework is effective for enriching pLMs.
- The proposed residue loss selection module enhances training by prioritizing high-quality and challenging structural learning signals, making the process more robust to noisy PDB data.
- Structural alignment, when carefully performed, can significantly improve a pLM's biophysical understanding without discarding its fundamental sequence modeling capabilities or increasing inference complexity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.