Reproduce “Composite Quantile Regression With XGBoost Using the Novel Arctan Pinball Loss” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Laurens Sluijterman, Frank Kreuwel, Eric Cator, Tom Heskes
Radboud University researchers developed the arctan pinball loss, a novel smooth approximation designed for XGBoost, facilitating efficient composite quantile regression within a single model. This method substantially reduces quantile crossings and improves the consistency of uncertainty estimates across diverse datasets.
Key insights
- A loss function's second derivative having a non-zero and polynomially decaying characteristic is crucial for effective second-order optimization in gradient boosting algorithms like XGBoost.
- Predicting multiple quantiles simultaneously within a single, multi-output model inherently promotes consistency and significantly reduces quantile crossings compared to training separate models for each quantile.
- Specific XGBoost hyperparameters, such as `min_child_weight` and `max_delta_step`, need careful adjustment to stabilize training and improve quantile consistency when using loss functions with non-constant second derivatives.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.