Reproduce “Spectrally Deconfounded Gradient Boosting” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Andrea Nava, Peter Bühlmann, Fabio Sigrist
Researchers from ETH Zürich introduced Spectrally Deconfounded Gradient Boosting, which extends spectral deconfounding principles to robustly estimate causal effects in the presence of hidden confounding for generalized likelihoods and nonlinear confounding. This method achieves reduced estimation error and more accurate feature importance compared to standard boosting, while also improving computational scalability over prior nonlinear spectral deconfounding techniques.
Key insights
- Deconfounding in gradient boosting results from the interaction of spectral shrinkage and regularization, which slows learning in confounding-aligned directions.
- An "intermediate stopping regime" is created, allowing true signals to be learned while confounding-driven patterns remain unfitted due to early stopping.
- A LAVA-type spectral transformation is equivalent to a whitening transformation derived from a linear mixed model, enabling data-driven tuning via empirical Bayes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.