Reproduce “Pinpointing Pivot Points: The DECIDE Framework for Data Analysis Decision Importances” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zachary T. Rewolinski, Jan Simson, Frauke Kreuter, Christoph Kern, Bin Yu
DECIDE measures importance of analytical choices in a multiverse by grouping analyses that differ in one choice, measuring outcome spread within each group, and summarizing across backgrounds. This direct comparison captures interactions without fitting a surrogate. In one real-data ablation, DECIDE’s rankings had mean Spearman correlation 0.91, a consistency measure rather than ground-truth validation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.