Reproduce “Data-Driven Subgroup Identification for Linear Regression” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zachary Izzo, Ruishan Liu, James Zou
DDGroup is an interpretable, data-driven method for identifying subgroups within datasets where a simple linear relationship between features and outcome holds. It recovers true regions in synthetic data and consistently achieved lower test Mean Squared Error than baseline methods on five real-world medical datasets, while also uncovering qualitatively opposite effects in a public health case study.
Key insights
- A robust local search for the minimum OLS MSE over k-nearest neighbor groups can effectively identify a 'core' of homogenous data, even amidst global heterogeneity, serving as an anchor for subgroup discovery.
- By defining rejection labels based on residuals from this locally derived model and a theoretically grounded threshold, it is possible to delineate points belonging to a true linear region from those that do not, with high probability of avoiding erroneous rejections.
- The problem of identifying the largest interpretable, axis-aligned region containing no rejected points can be efficiently solved by iteratively expanding a box from a center, ensuring computational tractability while maintaining desirable theoretical recovery properties.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.