Reproduce “TrialDesignBench: Benchmarking Language Models for Phase-I Trial Design Against Clinical Knowledge” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yihan Hank Tang, Peng Yang, Jiayi Xin, Tianqi Shang, Xuyang Chen, Ying Yuan, Yi Lian, Qi Long

TrialDesignBench benchmarks language models’ recommendations for 617 Phase-I and Phase-I/II oncology dose-finding trials against an expert-reviewed contemporary workflow. Across five models, adding inferred design factors raises average class-wise balanced accuracy from 0.307 to 0.531, while adding workflow instructions raises it to 0.547. These results evaluate agreement with one formalized workflow, not clinical optimality; no model matches its reference standard.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.