Reproduce “MLIP Detective: Active Failure Mode Discovery Beyond Benchmark Scores for Machine-Learning Interatomic Potentials” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ryuhei Okuno, Nontawat Charoenphakdee, Kaoru Hisama, Yuta Tsuboi

Preferred Networks developed the MLIP Detective framework, employing large language model agents to actively discover unphysical failure modes in machine-learning interatomic potentials (MLIPs) not captured by standard benchmarks. This method efficiently screens potential anomalies with inexpensive surrogate signals, identifying known training data inconsistencies and uncovering new, DFT-verified unphysical desorption pathways and mid-range attractions in tested MLIPs.

Key insights

  • Large language model (LLM) agents can effectively generate physics-informed test protocols and interpret simulation results for open-ended reasoning in atomistic simulations.
  • Inexpensive surrogate signals, specifically cross-model disagreement between models and violations of physical constraints, serve as reliable indicators to flag potential failure modes, guiding the active search.
  • A human-agent collaborative framework enables efficient discovery of anomalies by offloading initial exploration to agents while reserving human expertise for definitive, high-cost verification.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.