Reproduce “recAMPLIFY: Large-Scale Pretraining of Recursive Protein Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Saurav Jha, Qianli Liao, Robert Vernon, Sarath Chandar, Quentin Fournier

recAMPLIFY improves masked-residue protein prediction by repeatedly applying a shared Transformer block to residue-aligned representations, with learned gates controlling state updates and cycle contributions. In a 1M-step comparison trained on the same 2T tokens, its 155M parameters yield 69.6% fold accuracy, versus 56.8% for 354M AMPLIFY-350M, a 22.5% relative gain; this does not show faster inference.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.