Reproduce “Ensemble Methodology:Innovations in Credit Default Prediction Using LightGBM, XGBoost, and LocalEnsemble” on OpenResearch

About this paper

Mengran Zhu, Ye Zhang, Yulu Gong, Kaijuan Xing, Xu Yan, Jintong Song

An ensemble methodology combining LightGBM, XGBoost, and a novel LocalEnsemble module demonstrates enhanced accuracy in credit default prediction. The framework achieved a public score of 0.80128 and a private score of 0.80872 on the American Express - Default Prediction dataset, surpassing individual machine learning and deep learning models.

Key insights

• Strategic integration of diverse gradient boosting models within an ensemble, particularly when combined with distinct feature sets for each component, significantly amplifies predictive accuracy and generalization in complex financial datasets.
• Well-engineered gradient boosting ensembles continue to demonstrate superior performance for tabular time-series data compared to advanced deep learning architectures in credit default prediction tasks.
• Feature importance analysis revealed that a concentrated subset of features (top 50) contributed over 90% of the predictive power in both XGBoost and LightGBM, offering insights for model interpretability and potential efficiency gains.

Summary by alphaXiv.