Reproduce “Exploring Loss Design Techniques For Decision Tree Robustness To Label Noise” on OpenResearch

About this paper

Lukasz Sztukiewicz, Jack Henry Good, Artur Dubrawski

Researchers at Carnegie Mellon University investigated whether deep learning loss design techniques, including forward/backward loss correction and symmetric loss functions, could enhance the robustness of decision trees to label noise. Their work concluded that these methods do not yield practical improvements for decision trees or their ensembles due to the inherent learning mechanisms of trees.

Key insights

• Forward loss correction does not alter the fundamental splitting structure of decision trees, only modifying the predicted leaf values, and therefore provides no structural advantage against label noise.
• Symmetric loss functions are unsuitable for conventional decision tree induction because their optimal leaf values are plurality indicators, which often results in zero impurity gain and premature halting of tree growth.
• Backward loss correction, while theoretically capable of inducing a different tree structure by transforming input labels, did not translate into practical performance improvements in empirical testing.

Summary by alphaXiv.