Reproduce “AUTOCT: Automating Interpretable Clinical Trial Prediction with LLM Agents” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Fengze Liu, Haoyu Wang, Joonhyuk Cho, Dan Roth, Andrew W. Lo

Researchers at the University of Pennsylvania and MIT developed AUTOCT, an LLM agent-based framework that automates interpretable feature engineering and prediction for clinical trial outcomes. This system achieves competitive ROC-AUC scores, such as 0.753 for Phase I trial approval, while providing transparent insights and rigorously preventing label leakage by using a publication-date filter.

Key insights

  • Combining LLM-driven automated feature engineering with interpretable classical machine learning models provides both predictive power and transparency in high-stakes domains.
  • Explicitly filtering information retrieval based on publication dates effectively mitigates label leakage, ensuring robust and realistic clinical trial prediction performance.
  • An iterative, agent-based framework guided by Monte Carlo Tree Search can mimic and automate complex expert workflows for data-driven discovery.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.