Reproduce “Does Your Trial Simulator Know What the Drug Does? An Intervention-Blind Test of Interventional Validity on 7,207 Registered Trials” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Alicem Koyun
The authors test whether trial simulators use intervention details by comparing predictions from matched models with those details shown or withheld. On 878 binary treatment-comparator contrasts from 655 trials, registry-trained gradient boosting had interventional skill of −0.123, while Claude Sonnet 5 had 0.201. This measures incremental predictive value, not biological mechanism, and retained protocol clues may reveal treatment class.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.