Reproduce “Improving Deep Regression with Ordinal Entropy” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shihao Zhang, Linlin Yang, Michael Bi Mi, Xiaoxu Zheng, Angela Yao
Deep regression models often underperform classification on continuous tasks due to lower feature entropy; this work proposes an Ordinal Entropy regularizer that increases feature diversity while preserving ordinal relationships. The regularizer consistently improved performance across depth estimation, crowd counting, and age estimation, achieving new state-of-the-art results on NYU-Depth-v2 when integrated with NeW-CRFs.
Key insights
- Standard regression with MSE loss leads to feature representations with lower marginal entropy (less diversity) compared to classification with cross-entropy loss, even as it achieves feature tightness.
- Maximizing feature diversity (high marginal entropy H(Z)) is crucial for robust representation learning in continuous prediction tasks, complementing the need for feature compactness (low conditional entropy H(Z|Y)).
- Ordinal relationships inherent in continuous labels can be effectively leveraged to guide the learning of diverse feature spaces without requiring explicit label discretization.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.