Reproduce “NRGBoost: Energy-Based Generative Boosted Trees” on OpenResearch

About this paper

João Bravo

NRGBOOST is an energy-based generative boosting algorithm designed for tabular data, extending the principles of gradient-boosted trees to explicitly model data density. It achieves strong generative performance and competitive discriminative capabilities compared to state-of-the-art methods, while also enabling flexible inference and principled handling of missing data.

Key insights

• The principles of second-order gradient boosting can be successfully adapted to create powerful explicit density models for tabular data, bridging a gap between discriminative and generative tree-based methods.
• Formulating generative models as Energy-Based Models allows for explicit data density estimation, enabling versatile applications like flexible inference, out-of-distribution detection, and principled missing data handling.
• Computational challenges in EBM training, particularly MCMC sampling, can be mitigated through amortized sampling techniques, making boosting practical for generative tasks.

Summary by alphaXiv.