Reproduce “ProtDBench: A Unified Benchmark of Protein Binder Design and Evaluation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Cong Liu, Milong Ren, Jiaqi Guan, Chengyue Gong, Jinyuan Sun, Xinshi Chen, Wenzhi Xiao

Researchers from the University of Amsterdam and ByteDance Seed introduced ProtDBench, a unified, throughput-aware evaluation framework for *de novo* protein binder design. It reveals significant biases in current *in silico* evaluation practices due to verifier choice and highlights diverse trade-offs in efficiency, success rate, and structural diversity among generative models.

Key insights

  • The choice of structure prediction verifier significantly influences evaluation outcomes, with different verifiers exhibiting distinct inductive biases and recovering largely non-overlapping sets of true binders.
  • Practical performance in protein binder design involves complex trade-offs, as generation throughput and per-sequence success rate are often decoupled, and models vary in their ability to generate diverse structures.
  • AF3-style predictors generally show higher top-k enrichment on wet-lab data and lightweight variants like Protenix-Mini offer a better efficiency-accuracy balance compared to earlier AF2-style models.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.